Setting up an APT Mirror

April 6, 2017

Introductory Paragraph

Hello! Well, I have taken it upon myself to setup an APT mirror on site to speed up provisioning of machines here. I am working on setting up Foreman and decided that my iteration time was too slow. I’ll talk about the Foreman install in another post.

Anyway!

So you want to set up an APT mirror. When you looked around on Google you saw a bunch of posts from a long time ago that have failed to produce a working installation. If you look at enough of those posts, you can get an instance up BUT I will try to put it all in one place for you.

So why do you want an APT mirror?

Well, I am attempting to setup an automated machine provisioning system in order to allow myself and others to create machines more easily. You see, I am the only one who really uses linux machines in this shop and I would like to make it easier to manage them in case I get hit by a bus. The main reason I want the APT mirror on site is because our pipe to the internet is not really good enough for the number of users we have so downloads can be a bit slow. As the number of machines provisioned and number of installs increases, APT will begin to take up more and more of our bandwidth. Furthermore, I would like to speed up the iteration time for my foreman install as each ubuntu install currently takes ~45 minutes. I want the deployment process to be flawless before I present it to my team and right now I have to wait about an hour to test each tweak.

TL;DR: Ubuntu provisioning takes too long when pulling from an off-site mirror. Local mirror on 10G network is more better ™

So what do I need?

The two main pieces you need for this are disk space and time.

Alright, on with the steps now.

So here is the process I followed to get my install working.

  1. Setup an Ubuntu machine (This post is being written in the 16.04 LTS era)
  2. Install the following packages
    • apache2
    • apt-mirror
    • One liner: sudo apt-get install apache2 apt-mirror

  3. Configure your mirror The configuration file is located here: /etc/apt/mirror.list. I had to add i386 arch in order to get apt-get update to maybe work. This is what the file looks like:

    ############# config ##################
#
# set base_path    /var/spool/apt-mirror
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     20
set _tilde 0
#
############# end config ##############

deb-amd64 http://us.archive.ubuntu.com/ubuntu xenial main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu xenial-backports main restricted universe multiverse

deb-i386 http://us.archive.ubuntu.com/ubuntu xenial main restricted universe multiverse
deb-i386 http://us.archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse
deb-i386 http://us.archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse
deb-i386 http://us.archive.ubuntu.com/ubuntu xenial-backports main restricted universe multiverse

#Installers
deb-amd64 http://us.archive.ubuntu.com/ubuntu xenial main/debian-installer main restricted/debian-installer universe/debian-installer multiverse/debian-installer
deb-i386 http://us.archive.ubuntu.com/ubuntu xenial main/debian-installer main restricted/debian-installer universe/debian-installer multiverse/debian-installer

clean http://us.archive.ubuntu.com/ubuntu

  4. Enable the cron job for automatic updates There should have been a file created at /etc/crontab.d/apt-mirror My install came a with line already but if there isn’t one use: 0 4 * * * apt-mirror /usr/bin/apt-mirror > /var/spool/apt-mirror/var/cron.log

  5. Running apt-mirror for the first time. To run apt-mirror go ahead and run sudo apt-mirror

  6. Create a config file for apache: /etc/apache2/sites-available/10-aptmirror.conf Mine has the following content (Replace ServerAdmin and ServerName):

    <VirtualHost *:80>
ServerAdmin [YOUR EMAIL]
ServerName [SERVER FQDN]

DocumentRoot /var/www/apt
<Directory /var/www/apt>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

ErrorLog ${APACHE_LOG_DIR}/error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

CustomLog ${APACHE_LOG_DIR}/apt.access.log combined
</VirtualHost>

  7. Create symlinks

    • sudo ln -s /var/spool/apt-mirror/mirror/us.archive.ubuntu.com/ /var/www/apt
    • sudo ln -s /etc/apache2/sites-available/10-aptmirror.conf /etc/apache2/sites-enabled/10-aptmirror.conf

  8. Gotta reload da serber

    • sudo service apache2 reload

  9. Done!

Note: The apt-mirror download will happily saturate a 100meg pipe. Not recommended for use during business hours.

Note 2: SSL isn’t required as the checksum verification of APT ensures the integrity of the packages.

Setting Up Clients

In order for your servers to update from your APT mirror you have to update the sources APT uses.

The sources are defined in /etc/apt/sources.list. You will need to update the base domain of the apt repositories in order to your machine to update from the APT mirror.

You will see a variety of lines that look like the one below. Each of the lines corresponds to a particular package respository such as security or universe packages.

deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted

You should replace the portion between http:// and /ubuntu/ with the hostname of your webserver. Here’s an example:

deb http://apt.mgauto.me/ubuntu/ xenial main restricted

You can do this quickly if you open the file in vim.

  1. Backup your original sources file: sudo cp /etc/apt/sources.list /etc/apt/sources.list.orig
  2. Open the file in vim vim /etc/apt/sources.list
  3. Type :%s/[Original Hostname]/[Your Mirror's Hostname]/g and then press enter
    • Example: :%s/us.archive.ubuntu.com/apt.mgauto.me/g
  4. Type :wq and then press enter to save and exit

After you change your source file you can run sudo apt-get update to test your setup. You should see you mirror’s hostname in the output and the command should exit with no errors.

If you got the expected output then you are good to go! Congratulations! You setup your very own APT mirror.