Apt repository hosting

Aptly looks promising.

Had quirk with publishing to us-east-1 on S3: http://www.aptly.info/doc/feature/s3/. Bypassed by using eu-west-1. Troubleshooting S3 with GoLang was interesting :)

For a private repository, I borrowed an idea from here: http://skife.org/apt/aws/2012/10/12/private-apt-repos-in-s3.html (apt-transport-s3 - depended on cdbs on my machine, which was not mentioned in its docs).

Things I'm still unsure of:

• Automating publishing of packages with Aptly, and an easy way to control which packages are published.
• Safe distribution of the AWS Access and Secret keys to users. According to This they can also come as environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_KEY_ID.

* Add SSL to S3 on my own domain.

• It's recommended Here, instead of "precise" to have "dev", "staging" and "prod". (Some other good advice there as well: mirror useful packages to my own repository instead of counting on other people to keep it up. It simplifies keys as well. If other package's conf is needed - setup diversion).
• Maybe there's a simpler solution than using s3-apt-transport?
• Will CloudFront work OK? Caching can be a pain.
• How do I choose on github on which fork to fix the documentation of the apt-transport-s3 dependency?
• Publishing my public keys.

My aptly configuration and some commands:

{
  "rootDir": "/home/eburcat/.aptly",
  "downloadConcurrency": 4,
  "downloadSpeedLimit": 0,
  "architectures": [],
  "dependencyFollowSuggests": false,
  "dependencyFollowRecommends": false,
  "dependencyFollowAllVariants": false,
  "dependencyFollowSource": false,
  "gpgDisableSign": false,
  "gpgDisableVerify": false,
  "downloadSourcePackages": false,
  "ppaDistributorID": "ubuntu",
  "ppaCodename": "",
  "S3PublishEndpoints": {
    "eburcat.private":{
      "awsAccessKeyID":"",
      "awsSecretAccessKey":"",
      "region":"eu-west-1",
      "bucket":"repo.eburcat.com",
      "prefix":"private",
      "acl":"private",
      "encryptionMethod":"AES256"
    },
    "eburcat.public":{
      "awsAccessKeyID":"",
      "awsSecretAccessKey":"",
      "region":"eu-west-1",
      "bucket":"repo.eburcat.com",
      "prefix":"public",
      "acl":"public-read"
    }
  },
  "SwiftPublishEndpoints": {}
}

aptly repo create -distribution=precise -component=main eburcat-public
aptly repo add eburcat-public apt-transport-s3_1.1.1ubuntu2_amd64.deb
aptly repo add eburcat-public apt-transport-s3_1.1.1ubuntu2.dsc
aptly snapshot create eburcat-public-0.01 from repo eburcat-public
aptly publish snapshot eburcat-public-0.01 s3:eburcat.public:

aptly repo create -distribution=precise -component=main eburcat-release
aptly repo add eburcat-release my-package_1.0_amd64.deb
aptly snapshot create eburcat-0.01 from repo eburcat-release
aptly publish snapshot eburcat-0.01 s3:eburcat.private:

To publish my public key, I put it on S3, and then I can install it on any machine:

gpg --export --armor > /tmp/eburcat.pub
aws --region="eu-west-1" s3 cp --acl="public-read" /tmp/eburcat.pub s3://repo.eburcat.com/

To install a private package on a new machine:

wget -qO - https://s3-eu-west-1.amazonaws.com/repo.eburcat.com/eburcat.pub | sudo apt-key add -

# NOTE: https won't work with a proxy, use http instead
sudo echo "deb [arch=amd64] https://s3-eu-west-1.amazonaws.com/eburcat.com/public precise main" > /etc/apt/sources.list.d/eburcat.sources.list
sudo apt-get update
sudo apt-get install apt-transport-s3

# NOTE: Doesn't work with an apt proxy, you'll get:
# W: Failed to fetch https://s3-eu-west-1.amazonaws.com/repo.eburcat.com/public/dists/precise/main/binary-amd64/Packages 404 OK
sudo echo "deb [arch=amd64] s3://s3-eu-west-1.amazonaws.com/repo.eburcat.com/private precise main" >> /etc/apt/sources.list.d/eburcat.sources.list
sudo apt-get update
sudo apt-get install my-package