Difference between revisions of "How to cache openSUSE repositories with Squid"

From Perswiki
Jump to: navigation, search
(→‎Squid)
(→‎Overall/theoretical efficiency)
 
(50 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
'''Summary:''' how to make your local squid web cache work with openSUSE repositories and the openSUSE network installation process. In effect,  
+
==Summary==
−
how to run a fully autonomous local repository mirror.
+
How to make your local Squid web cache work with openSUSE repositories and the openSUSE network installation process. In effect,  
 +
how to run a fully autonomous, local on-demand repository mirror.  Even with a high-speed ADSL internet connection, savings of up to 60% are easily achieved.
 +
 
 +
In 2012, I wrote this for squid 2.7. In April 2016, I updated everything to work with squid 3.4.x.
 +
 
 +
==Before you begin==
 +
Unless you already have a working squid web-cache, or you want to set it up anyway, the setup I describe here is probably too complex. It will probably be easier to run a local rsync-mirror.
  
 
== Background ==
 
== Background ==
Line 10: Line 16:
 
as such, but because we might be testing or debugging our own add-ons or to collect diagnostics.  
 
as such, but because we might be testing or debugging our own add-ons or to collect diagnostics.  
 
Sometimes we install on virtual machines, sometimes on desktops, more often on
 
Sometimes we install on virtual machines, sometimes on desktops, more often on
−
server hardware in our downstairs datacentre.  We have a local squid web cache, but after having switched to doing network installs more frequently, I have often been annoyed by the lack of effectiveness for caching the openSUSE repository. When I've already done one installation, the downloads for a subsequent one should obviously happen a lot faster, in fact at wire speed.
+
server hardware in our downstairs datacentre.  We have a local Squid web cache, but after having switched to doing network installs more frequently, I have often been annoyed by the lack of effectiveness for caching the openSUSE repository. When I've already done one installation, the downloads for a subsequent one should obviously happen a lot faster, in fact at wire speed.
−
Well, they don't and that's annoying when you know they should have been cached.
+
Well, they don't and that's annoying when you know they could have been cached.
  
−
The immediate alternative would be to run a local copy of the openSUSE repositories, but it requires a process for keeping a the local mirror up-to-date, plus a bit of manual interaction (adding the right URL when installing. This is all entirely feasible, but I thought using squid would be a more elegant and (hopefully) fully autonomous solution. so I decided to figure out why our squid wasn't coping.  
+
The immediate alternative would be to run a local mirror of the openSUSE repositories, but it requires a process for keeping a the local mirror up-to-date, plus a bit of manual interaction (adding the right URL when installing. This is all entirely feasible, but I thought using Squid would be a more elegant and (hopefully) fully autonomous solution. so I decided to figure out why our Squid wasn't coping.  
  
−
Well, squid and the openSUSE network installation process just don't work together very well. Not out-of-the-box anyway.  
+
Well, Squid and the openSUSE network installation process just don't work together very well. Not out-of-the-box anyway.  
−
The repository at [http://download.opensuse.org download.opensuse.org] is served by a load-distribution system combining mirrorbrain and metalinks. I won't
+
The repository at [http://download.opensuse.org download.opensuse.org] is served by a load-distribution system combining [http://www.mirrorbrain.org mirrorbrain] and [http://www.metalinker.org/ metalinks].  
−
go into any further detail, suffice to say that this means packages are downloaded using [http://en.wikipedia.org/wiki/Segmented_downloading segmented downloading] spread over multiple mirrors, which together makes it impossible for squid to do much caching.
+
I won't go into any further detail, suffice to say that this means packages are downloaded using [http://en.wikipedia.org/wiki/Segmented_downloading segmented downloading] spread over multiple mirrors, which together makes it impossible for squid to do much caching.
  
 
== The problem ==
 
== The problem ==
−
Well, two problems really:  
+
Well, three problems really:  
  
−
* the openSUSE repository is mirrored around the world. Mirrorbrain does a good job of picking the most suitable mirrors depending on your location, which also means a good distribution so individual mirrors aren't overloaded. However, squid does not know that multiples mirror sites serve the same file, so caching is rendered largely ineffective.  
+
* the openSUSE repositories are mirrored around the world, clients are served by Mirrorbrain. Mirrorbrain does a good job of picking the most suitable IPv4 mirrors depending on your location, which presumably also means a good distribution so individual mirrors aren't overloaded. However, Squid does not know that multiple mirror sites serve the same file, making caching at best ineffective.  
−
* the segmented download means a package is downloaded in bits from multiple mirrors. This is good for speeding up the download and making good use of the available downstream bandwidth. The problem is that squid is only able to cache whole files, not parts of files, so now caching is completely useless.
+
* the segmented download means a package is downloaded in bits from multiple mirrors. This is good for speeding up the download and for maximizing the use of the available downstream bandwidth. The problem is that Squid is only able to cache whole files, not parts of files, rendering caching completely useless.
 +
* at the time of writing, the openSUSE mirror setup had a poor IPv6 geolocation. If you are accessing via IPv6, you are redirected to arbitrary mirrors around the world, some of them very slow.
  
−
I have solved both of these problems:  
+
I have solved all three problems:  
  
−
* using a squid url rewriter, I map all the mirror locations on to a single one.
+
* using a Squid url rewriter, I map all the mirror locations on to a single one.
−
* using a squid logfile and a custom written daemon, I do complete downloads of all the files that are being fetched with segmented downloading.
+
* using a Squid logfile and a custom written daemon, I do complete downloads of all the files that are being fetched with segmented downloading.
  
 
== Summary ==
 
== Summary ==
Line 40: Line 47:
  
 
Others doing e.g. repeated updates or adding software, should enjoy similar benefits (once the packages have been cached).
 
Others doing e.g. repeated updates or adding software, should enjoy similar benefits (once the packages have been cached).
 +
 +
== 60% faster at 6Mbit/s downstream ==
 +
I run this setup primarily to save time on installations.  In the office, we have a 6000/600Kbit ADSL connection.  It's sufficient for most
 +
activities, but when installing openSUSE over the network, it's really a bit slow.  For openSUSE 12.1, it takes about an hour to complete phase 1 of the install process - 6-7 minutes
 +
for the initial 6 system installation images, then 50 minutes for a vanilla KDE installation.
 +
 +
However, installing at wire speed (our LAN is 100Mbit) from the Squid cache is a lot faster, taking only 22 minutes (15 seconds for the initial 6 installation images, 21 minutes
 +
for phase 1 to complete.  That is a reduction of more than 60%.  With a slower network connection and perhaps slower mirrors too, only more time saved.
 +
 +
== Overall/theoretical efficiency ==
 +
'''Update 2019/06/16.'''  Since I originally wrote this article, the internet speeds available to consumers have been improving every other month.  My measly 6Mbit ADSL line is way outdated by now, but it is a good example.
 +
Your potential gains from implementing the scheme I have described here depend entirely on
 +
 +
* your internet downlink speed
 +
* your local network speed
 +
* your usage patterns
 +
 +
With my 2012 example, I had a 6 MBit/s downlink and 100Mbit/s on local ethernet.  Theoretically that made for a factor 16 improvement.  Today I have 1Gbit/s local ethernet and 1Gbit/s downlink, so theoretically no improvement.  I.e. a download from a local cache is no faster than a download from the internet.
 +
 +
For my personal/business use or benefit, the use case has all but disappeared.  Our downlink is plenty for infrequent single installs and we maintain a [http://opensuse.hostsuisse.net public mirror] too.  We don't do so many regular installs either, so the main saving is on the load on mirrors, which is probably negligible anyway.  (unless thousands of users were to do local caching with this scheme).
 +
 +
However, in a nutshell, if a local cache is faster than your internet access, you should still a worthwhile improvement using this scheme.
  
 
== Download ==
 
== Download ==
Line 45: Line 74:
 
up with the list of openSUSE mirrors. It's not as easy as just plonking another package into your openSUSE system with YaST or zypper, but the following step by step guide will hopefully help.
 
up with the list of openSUSE mirrors. It's not as easy as just plonking another package into your openSUSE system with YaST or zypper, but the following step by step guide will hopefully help.
  
−
[http://files.jessen.ch/fetcher206-1.0.tar.gz fetcher206-1.0.tar.gz]
+
[http://files.jessen.ch/fetcher206-1.3.tar.gz fetcher206-1.3.tar.gz]
  
−
Current version is 1.0.
+
Current version is 1.3.
  
−
== 16% faster at 50Mbit/s downstream ==
+
===Change history===
−
I run this setup primarily to save time on installations. In April 2012, our datacentre internet connection was upgraded 50Mbit/s fibre, more than plenty for a very fast network install.  
+
  2016/04/19 version 1.3
−
(with my current provider and their infrastructure, I don't think an openSUSE network install will be able to saturate it).  Prior to that we had only 6Mbit/s ADSL downstream, and the time savings with this setup were significantly higher.
+
Updates to work with Squid 3.4.x, added a fetcher206 systemd service file, updated jesred to 1.4.
−
 
+
   
−
Regardless, it's still faster installing at wire speed than downloading from a collection of mirrors.  At wire speed (my LAN is 100Mbit), a regular install of openSUSE 12.1 takes about 25 minutes (wall-clock time).  At 50Mbit/s it takes about 30 minutes, so approximately 16% saved with the proxy solution.  With a slower network connection and perhaps slower mirrors too, only more time saved.
+
2016/04/15 version 1.2a
 +
Tiny syntax error correction, no change in version#.  Thank you, hpj.
 +
 +
2015/05/21 version 1.2
 +
  Miscellaneous updates from the last three years.  
 +
   
 +
2012/06/04 version 1.1
 +
Improved parsing of the mirror list
 +
 +
2012/05/18 version 1.0
 +
First public release
  
 
== Step by step ==
 
== Step by step ==
 
===Squid===
 
===Squid===
−
The Squid web-proxy is the key element in this setup, so step one is setting up a working Squid installation.  
+
The Squid web-proxy is the key element in this setup, so a working Squid installation is prerequisite.
−
 
+
−
'''Note:''' the setup here works for Squid 2.7, I don't think the storeurl_rewrite feature has been implemented in Squid 3.x yet.
+
−
 
+
 
Setting up Squid is not as complicated as it may appear, but you'll have to consult squid documentation, it's outside the scope of this article.  
 
Setting up Squid is not as complicated as it may appear, but you'll have to consult squid documentation, it's outside the scope of this article.  
 
Whether you prefer directing access using environment variables ''http_proxy'' et al, or if you run a transparent proxy (like I do), is
 
Whether you prefer directing access using environment variables ''http_proxy'' et al, or if you run a transparent proxy (like I do), is
Line 67: Line 103:
 
===jesred===
 
===jesred===
 
''jesred'' is the URL rewriter. It's fairly mature, but fully functional. ([http://www.linofee.org/~jel/webtools/jesred/ original webpage]).  I had to make a couple of changes  
 
''jesred'' is the URL rewriter. It's fairly mature, but fully functional. ([http://www.linofee.org/~jel/webtools/jesred/ original webpage]).  I had to make a couple of changes  
−
to make it fully compatible with [http://www.squid-cache.org/ squid 2.7]:
+
to make it fully compatible with [http://www.squid-cache.org/ squid 2.7], this was version 1.3. In April 2016, I made some minor changes to make it work with IPv6 clients too, version 1.4:
  
−
* [http://files.jessen.ch/jesred-1.3.tar.gz jesred-1.3.tar.gz]  
+
* [http://files.jessen.ch/jesred-1.4.tar.gz jesred-1.4.tar.gz]  
  
 
For the moment, it does not come packaged, you'll have to build it from scratch:
 
For the moment, it does not come packaged, you'll have to build it from scratch:
  
 
  tar xzvf <tarball>
 
  tar xzvf <tarball>
−
  cd jesred-1.3
+
  cd jesred-1.4
 
  make
 
  make
  
−
Installation: when you're done, copy the binary ''jesred'' into /usr/bin.
+
Installation: when you're done, copy the binary ''jesred'' into /usr/local/bin or whatever your preferred location for your own binaries is.
  
 
The config file for jesred: ''/etc/squid/jesred.conf''
 
The config file for jesred: ''/etc/squid/jesred.conf''
Line 95: Line 131:
 
Configuration: add the following lines to /etc/squid/squid.conf
 
Configuration: add the following lines to /etc/squid/squid.conf
  
−
  storeurl_rewrite_program /usr/bin/jesred
+
  store_id_program /usr/local/bin/jesred
−
  storeurl_rewrite_children 5
+
  store_id_children 5
 
   
 
   
 
  acl metalink req_mime_type application/metalink4+xml
 
  acl metalink req_mime_type application/metalink4+xml
−
  storeurl_access deny metalink  
+
  store_id_access deny metalink  
−
   
+
  store_id_access allow localnet
−
storeurl_access allow localnet
+
 
−
storeurl_access allow localhost
+
I also recommend changing "maximum_object_size" to e.g. 128Mb (default is 4Mb).
−
+
−
acl localhost src 127.0.0.0/8
+
−
acl localnet src 192.168.0.0/16
+
  
 
===fetcher206 logfile===
 
===fetcher206 logfile===
−
Amend ''/etc/squid/squid./conf'' as follows:
+
Amend ''/etc/squid/squid.conf'' as follows:
  
 
  logformat f206 %{%Y-%m-%dT%H:%M:%S}tl %Ss/%03Hs %rm %ru %mt
 
  logformat f206 %{%Y-%m-%dT%H:%M:%S}tl %Ss/%03Hs %rm %ru %mt
Line 149: Line 182:
  
 
===mirror database===
 
===mirror database===
−
We need a current list of the available openSUSE mirrors. This can be retrieved from mirrors.opensuse.org.  
+
We need a current list of the available openSUSE mirrors. This can be retrieved from mirrors.opensuse.org. For the time being, I use XSL to
 +
parse the HTML page, but I hope to move to a suitably formatted list direct from MirrorBrain.
 +
 
 
  mkdir -p /var/lib/fetcher206
 
  mkdir -p /var/lib/fetcher206
 
  cp ''tarball/Makefile.mirrors'' /var/lib/fetcher206/Makefile
 
  cp ''tarball/Makefile.mirrors'' /var/lib/fetcher206/Makefile
 +
cp ''tarball/extract*'' /var/lib/fetcher206/
 
  make -C /var/lib/fetcher206
 
  make -C /var/lib/fetcher206
 
  cp ''tarball/opensuse_mirrors.cron'' /etc/cron.d/opensuse_mirrors
 
  cp ''tarball/opensuse_mirrors.cron'' /etc/cron.d/opensuse_mirrors
 +
 +
April 2016: I have not yet found a way of getting a mirror list straight from mirrorbrain.
  
 
===reload squid===
 
===reload squid===
Line 162: Line 200:
  
 
===fetcher206===
 
===fetcher206===
−
''fetcher206'' is, for the time being, a PHP script. Install it by simply copying it into /usr/bin.
+
''fetcher206'' is, for the time being, a PHP script. Install it by simply copying it into /usr/local/bin.
 
It has a few hard-coded options, such as number of wgets to run concurrently, name of logfile etc.
 
It has a few hard-coded options, such as number of wgets to run concurrently, name of logfile etc.
  
−
fetcher206 does not yet have a systemd service unit, nor an LSB init-script. For the time being, you
+
fetcher206 comes with a systemd service unit, see the tar file.
−
simply start it with:
+
  
−
startproc -s -q /usr/bin/fetcher206
+
fetcher206 needs a couple of extra PHP modules - php5-pcntl and php5-xsl.

Latest revision as of 08:08, 17 June 2019

Summary

How to make your local Squid web cache work with openSUSE repositories and the openSUSE network installation process. In effect, how to run a fully autonomous, local on-demand repository mirror. Even with a high-speed ADSL internet connection, savings of up to 60% are easily achieved.

In 2012, I wrote this for squid 2.7. In April 2016, I updated everything to work with squid 3.4.x.

Before you begin

Unless you already have a working squid web-cache, or you want to set it up anyway, the setup I describe here is probably too complex. It will probably be easier to run a local rsync-mirror.

Background

In my company, we do quite a lot of testing of openSUSE, and over the last three-four years, we have increasingly switched to installing over the network. Prior to that, we would install from DVD images over NFS served by a local server. However, over last couple of years, we've been working a lot more with Factory and the regular snap-shots that lead up to a final/gold release. With those it is much easier to just point the installation process to the right URL and have everything downloaded there and then.

When we're testing installation or new hardware, we often have to repeat the installation process many times on different machines. Not because it doesn't work as such, but because we might be testing or debugging our own add-ons or to collect diagnostics. Sometimes we install on virtual machines, sometimes on desktops, more often on server hardware in our downstairs datacentre. We have a local Squid web cache, but after having switched to doing network installs more frequently, I have often been annoyed by the lack of effectiveness for caching the openSUSE repository. When I've already done one installation, the downloads for a subsequent one should obviously happen a lot faster, in fact at wire speed. Well, they don't and that's annoying when you know they could have been cached.

The immediate alternative would be to run a local mirror of the openSUSE repositories, but it requires a process for keeping a the local mirror up-to-date, plus a bit of manual interaction (adding the right URL when installing. This is all entirely feasible, but I thought using Squid would be a more elegant and (hopefully) fully autonomous solution. so I decided to figure out why our Squid wasn't coping.

Well, Squid and the openSUSE network installation process just don't work together very well. Not out-of-the-box anyway. The repository at download.opensuse.org is served by a load-distribution system combining mirrorbrain and metalinks. I won't go into any further detail, suffice to say that this means packages are downloaded using segmented downloading spread over multiple mirrors, which together makes it impossible for squid to do much caching.

The problem

Well, three problems really:

  • the openSUSE repositories are mirrored around the world, clients are served by Mirrorbrain. Mirrorbrain does a good job of picking the most suitable IPv4 mirrors depending on your location, which presumably also means a good distribution so individual mirrors aren't overloaded. However, Squid does not know that multiple mirror sites serve the same file, making caching at best ineffective.
  • the segmented download means a package is downloaded in bits from multiple mirrors. This is good for speeding up the download and for maximizing the use of the available downstream bandwidth. The problem is that Squid is only able to cache whole files, not parts of files, rendering caching completely useless.
  • at the time of writing, the openSUSE mirror setup had a poor IPv6 geolocation. If you are accessing via IPv6, you are redirected to arbitrary mirrors around the world, some of them very slow.

I have solved all three problems:

  • using a Squid url rewriter, I map all the mirror locations on to a single one.
  • using a Squid logfile and a custom written daemon, I do complete downloads of all the files that are being fetched with segmented downloading.

Summary

For anyone, an individual or a group of people, doing repeated ad-hoc installations of openSUSE (typically Factory), using this squid setup means

  • significantly faster installation due to downloads at wire speed
  • significant bandwidth savings due to a working cache
  • less load on openSUSE mirrors due to a working cache
  • zero local mirror management (assuming a working squid setup).
  • no need to worry about where to install from

Others doing e.g. repeated updates or adding software, should enjoy similar benefits (once the packages have been cached).

60% faster at 6Mbit/s downstream

I run this setup primarily to save time on installations. In the office, we have a 6000/600Kbit ADSL connection. It's sufficient for most activities, but when installing openSUSE over the network, it's really a bit slow. For openSUSE 12.1, it takes about an hour to complete phase 1 of the install process - 6-7 minutes for the initial 6 system installation images, then 50 minutes for a vanilla KDE installation.

However, installing at wire speed (our LAN is 100Mbit) from the Squid cache is a lot faster, taking only 22 minutes (15 seconds for the initial 6 installation images, 21 minutes for phase 1 to complete. That is a reduction of more than 60%. With a slower network connection and perhaps slower mirrors too, only more time saved.

Overall/theoretical efficiency

Update 2019/06/16. Since I originally wrote this article, the internet speeds available to consumers have been improving every other month. My measly 6Mbit ADSL line is way outdated by now, but it is a good example. Your potential gains from implementing the scheme I have described here depend entirely on

  • your internet downlink speed
  • your local network speed
  • your usage patterns

With my 2012 example, I had a 6 MBit/s downlink and 100Mbit/s on local ethernet. Theoretically that made for a factor 16 improvement. Today I have 1Gbit/s local ethernet and 1Gbit/s downlink, so theoretically no improvement. I.e. a download from a local cache is no faster than a download from the internet.

For my personal/business use or benefit, the use case has all but disappeared. Our downlink is plenty for infrequent single installs and we maintain a public mirror too. We don't do so many regular installs either, so the main saving is on the load on mirrors, which is probably negligible anyway. (unless thousands of users were to do local caching with this scheme).

However, in a nutshell, if a local cache is faster than your internet access, you should still a worthwhile improvement using this scheme.

Download

For the impatient, I've tar'ed everything into a single download. This contains the daemon code, one sample config files and the scripts for keeping up with the list of openSUSE mirrors. It's not as easy as just plonking another package into your openSUSE system with YaST or zypper, but the following step by step guide will hopefully help.

fetcher206-1.3.tar.gz

Current version is 1.3.

Change history

2016/04/19 version 1.3
Updates to work with Squid 3.4.x, added a fetcher206 systemd service file, updated jesred to 1.4. 

2016/04/15 version 1.2a
Tiny syntax error correction, no change in version#.  Thank you, hpj.

2015/05/21 version 1.2
Miscellaneous updates from the last three years. 

2012/06/04 version 1.1
Improved parsing of the mirror list

2012/05/18 version 1.0 
First public release

Step by step

Squid

The Squid web-proxy is the key element in this setup, so a working Squid installation is prerequisite. Setting up Squid is not as complicated as it may appear, but you'll have to consult squid documentation, it's outside the scope of this article. Whether you prefer directing access using environment variables http_proxy et al, or if you run a transparent proxy (like I do), is not really important.

jesred

jesred is the URL rewriter. It's fairly mature, but fully functional. (original webpage). I had to make a couple of changes to make it fully compatible with squid 2.7, this was version 1.3. In April 2016, I made some minor changes to make it work with IPv6 clients too, version 1.4:

For the moment, it does not come packaged, you'll have to build it from scratch: 

tar xzvf <tarball>
cd jesred-1.4
make

Installation: when you're done, copy the binary jesred into /usr/local/bin or whatever your preferred location for your own binaries is.

The config file for jesred: /etc/squid/jesred.conf 

allow = /etc/squid/redirector.acl
rules = /etc/squid/opensuse-redirect.rules
redirect_log = /var/log/squid/redirect.log
rewrite_log = /var/log/squid/rewrite.log

Using /etc/squid/redirector.acl you can control which clients' requests the rewriter should process, but I find this is actually easier to control with Squid's ACL and storeurl_access directive, so I enable for all clients: 

# rewrite all URLs from
0.0.0.0/0

/etc/squid/squid.conf

Configuration: add the following lines to /etc/squid/squid.conf 

store_id_program /usr/local/bin/jesred
store_id_children 5

acl metalink req_mime_type application/metalink4+xml
store_id_access deny metalink 
store_id_access allow localnet

I also recommend changing "maximum_object_size" to e.g. 128Mb (default is 4Mb).

fetcher206 logfile

Amend /etc/squid/squid.conf as follows: 

logformat f206 %{%Y-%m-%dT%H:%M:%S}tl %Ss/%03Hs %rm %ru %mt
access_log /var/log/squid/fetch206.log f206

This log will be read by fetcher206.

To prevent it growing too big, add the following to /etc/logrotate.d/ : 

/var/log/squid/fetch206.log {
   compress
   dateext
   maxage 365
   rotate 5
   size=+4M
   notifempty
   missingok
   create 640 squid root
   sharedscripts
   postrotate
    /etc/init.d/squid reload
   endscript
}

squid delay pool

This is an optional step - depending on your available downstream bandwidth, you may want to restrict what is used by fetcher206 for retrieving the repository files. This prevents

  • slowing down the current installation and
  • abuse of the internet connection
delay_pools 1
delay_class 1 1
delay_access 1 allow localhost
delay_parameters 1 1000000/1000000

Add the above to /etc/squid/squid.conf - it defines one delay_pool, only accessible from localhost (which is where fetcher206 will be running wget) with a maximum bandwidth of 1MByte/sec.

If you have other http/proxy traffic originating from localhost, you could just add another 127.0.0.x address, and use that specifically for fetcher206.

mirror database

We need a current list of the available openSUSE mirrors. This can be retrieved from mirrors.opensuse.org. For the time being, I use XSL to parse the HTML page, but I hope to move to a suitably formatted list direct from MirrorBrain. 

mkdir -p /var/lib/fetcher206
cp tarball/Makefile.mirrors /var/lib/fetcher206/Makefile
cp tarball/extract* /var/lib/fetcher206/
make -C /var/lib/fetcher206
cp tarball/opensuse_mirrors.cron /etc/cron.d/opensuse_mirrors

April 2016: I have not yet found a way of getting a mirror list straight from mirrorbrain.

reload squid

When you've come this far, it's time to reload squid with 

squid -k reconfigure

fetcher206

fetcher206 is, for the time being, a PHP script. Install it by simply copying it into /usr/local/bin. It has a few hard-coded options, such as number of wgets to run concurrently, name of logfile etc.

fetcher206 comes with a systemd service unit, see the tar file.

fetcher206 needs a couple of extra PHP modules - php5-pcntl and php5-xsl.

Retrieved from "https://wiki.jessen.ch/index?title=How_to_cache_openSUSE_repositories_with_Squid&oldid=710"