/usr/share/doc/apt-cacher-ng/html/config-serv.html is in apt-cacher-ng 2-2.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=US-ASCII">
<title>Advanced Server Configuration</title>
<link rel="previous" href="config-servquick.html">
<link rel="ToC" href="index.html">
<link rel="up" href="index.html">
<link rel="next" href="secure.html">
</head>
<body>
<p><a href="config-servquick.html">Previous</a> | <a href="index.html">Contents</a> | <a href="secure.html">Next</a></p>
<ul>
<li><a href="#config-serv">Chapter 4: Advanced Server Configuration</a>
<ul>
<li><a href="#Vocabulary">4.1 Vocabulary</a></li>
<li><a href="#cfgfiles">4.2 Configuration file types</a></li>
<li><a href="#repmap">4.3 Repositories and URL mapping</a>
<ul>
<li><a href="#basic-map">4.3.1 Writing Remap-... configuration</a></li>
<li><a href="#remap-trickz">4.3.2 Special tricks and additional notes</a></li>
</ul></li>
</ul></li>
</ul>
<h1><a name="config-serv"></a>Chapter 4: Advanced Server Configuration</h1>
<h2><a name="Vocabulary"></a>4.1 Vocabulary</h2>
<p>
This chapter introduces some terminology which is needed to understand the functionality of apt-cacher-ng; it's recommended to understand it before continuing with the advanced configuration.
</p>
<ul><li>
"Backend": a text file consisting of a list of mirror URLs, one per line (a more complex RFC822-like format is also supported). Used for URL remapping; see <a href="#repmap">section 4.3</a>.
</li>
<li>
"Volatile files": files with changing contents on the server while keeping the same URL (i.e. the same filename and location). For example, metadata pertaining to package files stored in a remote archive is classified as 'volatile'. They are usually 'index files' known as Packages, Sources, Release, Pdiff and the like, and maybe also associated extra data (like translations and thumbnail icons).
</li>
<li>
"Package files": files that contain software packages and other "solid" data: DEBs, source files for their creation (.tar.gz, .diff, .dsc), various metadata which is not subject to change after first appearance on the server.
</li>
<li>
"Configuration line": one single line in the configuration file. Some examples in this chapter may be displayed as wrapped lines but should be stored as a single line in the configuration. Some variables (like <code>RequestAppendix</code>) support multiline data which needs to be separated with \n delimiter.
</li>
</ul>
<h2><a name="cfgfiles"></a>4.2 Configuration file types</h2>
<p>
By default, the /etc/apt-cacher-ng directory (or the one specified with program options) contains all config files, HTML page templates, the stylesheet and other text-based support files used by apt-cacher-ng. The contents may vary depending on the installation of apt-cacher-ng, refer to the package documentation for Linux Distribution packages.
</p>
<p>
There are a few certain file types distinguished by apt-cacher-ng:
</p>
<ol><li>
Main configuration files:
<p>
*.conf files are assumed to contain configuration directives in the form of "key: value" pairs. The package comes with a commented example configuration file. apt-cacher-ng reads all files matching *.conf in alphabetical order and merges the contents. For options documentation, see commented example file shipped with apt-cacher-ng (conf/ directory in original source).
</p>
<p>
For security reasons, the files can be made readable only to the daemon and administorator accounts, e.g. when they contain passwords or other sensitive data.
</p>
</li>
<li>
URL lists and remote repository list files. The file names are arbitrary, no special suffix is required. They are read and included during processing of configuration files and can contain data in one of the following formats:
<ul><li>
simple text files with one URL per line (the URL should point to the base directory of the repository, e.g. "http://ftp.de.debian.org/debian/"). A URL must start with http:// and should end with a slash
</li>
<li>
an RFC822-like format, with lines like 'Site: <hostname>' and 'Archive-http: /base/directory/of/repository/'. Optional fields are also used in this remapping descriptions to add more possible variants (Alias, Aliases, X-Archive-http:) of the URLs to the lookup list
</li>
</ul>
</li>
<li>
Various support files used for the configuration web interface, named like *.css and *.html.
</li>
<li>
*.default files are used in some rare cases as replacement for list files having the same name without .default suffix.
</li>
<li>
*.hooks files specify custom actions which can be executed upon connection/disconnection (see <a href="#remap-trickz">section 4.3.2</a> for details).
</li>
</ol>
<p>
Except from .conf files, most files listed above can be moved to another "support" directory and the daemon will look for them therein if they are not present in the primary configuration directory. This feature is intended to help keeping pure configuration data and mostly static data in different locations. The directory path is specified at build time and can be overriden with the <em>SupportDir</em> directive (and if used, this should be set as early as possible).
</p>
<h2><a name="repmap"></a>4.3 Repositories and URL mapping</h2>
<p>
With the most simple configuration, apt-cacher-ng will act almost like an ordinary HTTP proxy with improved caching behaviour. When files are requested, they are downloaded from a remote location specified in client's request and are stored in a unique way.
</p>
<p>
However, for some use cases it can be beneficial to specify additional rules to achieve further improvements, e.g. in order to detect and prevent avoidable downloads, to reduce space requirements for the cache directory or simply hide real download locations from the APT clients.
</p>
<p>
These modifications are generally achieved by two strategies, <code>Merging</code> and <code>Redirection</code>, which are configured in a context of a specified cache <code>Repository</code>. The configuration for them is created using one or multiple Remap-... configuration directives (see below).
</p>
<p>
<em>Merging:</em>
</p>
<p>
"Merging" of incoming requests can be done if some subdirectories of different remote servers are considered equal where the last part of the remote file path leads to the same file content. When specified, the internal cache content is shared and the live download stream is shared. The configuration work consists of setting an "equality list" containing a set of URLs representing the base directories (like <code>http://ftp.debian.org/debian</code> and <code>http://ftp.uni-kl.de/pub/linux/debian</code>).
</p>
<p>
<em>Redirection:</em>
</p>
<p>
With redirection, client requests cause a download from a remote location which is different from what clients requested and believe to receive from. Redirection is an optional feature; if used, it's configured by one or multiple URL(s) pointing to target servers. The URL(s) must include a directory spec which matches the directory level of the URLs in the <code>Merging</code> URL(s), for example all ending with /ubuntu/ for usual Ubuntu mirror URLs. If redirection is not used (i.e. the target URL list is empty) the original URL from client's request is used to get the data.
</p>
<p>
<em>Repository:</em>
</p>
<p>
A (cache) repository is the internal identifier which declares the scope in which <code>Merging/Redirection</code> specs are applied. It also represents the name of an internal cache subdirectory.
</p>
<h3><a name="basic-map"></a>4.3.1 Writing Remap-... configuration</h3>
<p>
When use cases for merging/redirection are identified and a repository name is chosen, these components are written into configuration directives starting with Remap- which follow the simple syntax:
</p>
<p>
Remap-<code>RepositoryName</code>: <code>MergingURLs</code> ; <code>TargetURLs</code> ; <code>OptionalFlags</code>
</p>
<p>
The repository name is a symbolic name which should be chosen carefully and should not be changed afterwards, otherwise the data might become inaccessible for clients until the files are extracted and reimported semi-manually. Internally, this string shares the namespace with host names and/or top directory names of other URLs. Name collisions can cause nasty side effects and should be avoided. Recommended names are made up from alphanumeric or URL-friendly characters. Also, a repository name should not be associated to a real hostname. Examples for good names: <code>archlinux-repo</code>, <code>debianlocal</code>. Examples for bad names: <code>fedora.example.com</code>, <code>_very&weird</code>.
</p>
<p>
The <code>TargetURLs</code> part is optional (see <code>Redirection</code> description above). If multiple targets are specified, the order of servers here defines their order of preference (see also the <code>NetworkTimeout</code> option and additional notes below).
</p>
<p>
Both URL lists simply contain URLs separated by spaces. The strings must be properly URL-encoded. Since all URLs are assumed to belong to http:// protocol and point to a remote directory, the <code>http://</code> protocol prefix and trailing slashes are optional. There is no hard limit to the number of URLs. However, for readability reasons it's recommended to put them into separate list files (see <a href="#cfgfiles">section 4.2</a>) and specify the particular list files with tags like <code>file:urlsDebian.list</code> instead of writing them into a single line. Raw URLs and <code>file:...</code> lists can be mixed.
</p>
<p>
Fully configured Remap lines can look like:
</p>
<p>
<em>Example I:</em>
</p>
<p>
<code>Remap-debrep: ftp.de.debian.org/debian http://ftp.at.debian.org/debian</code>
</p>
<p>
for the use case: small home network, clients have de... or at... servers in their sources.list files and use acng as HTTP proxy. Now the files are still downloaded from at... or de... mirrors depending on the user request, but already cached data is served to both, at... and de... users.
</p>
<p>
<em>Example II:</em>
</p>
<p>
<code>Remap-ubuntu: file:ubumir.lst ; 192.168.17.23/pu ca.archive.ubuntu.com/ubuntu</code>
</p>
<p>
for the use case: small home network, clients have various Ubuntu mirrors (which are listed in ubumir.lst) in their sources.list files and use acng as HTTP proxy. All requests are redirected to a mirror in the /pu directory of some local machine. When that machine is down, Canadian public server is used instead.
</p>
<h3><a name="remap-trickz"></a>4.3.2 Special tricks and additional notes</h3>
<p>
There are some implementation details (partially explained above) and some configuration options related to repository settings which should be mentioned explicitly.
</p>
<p>
The internal cache directory tree follows the URL requests from the clients unless modified by Remapping rules. For proxy-style configuration on the user side, it is always the hostname of the requested URL. But if clients access the apt-cacher-ng server like a regular mirror (not using APT's proxy config) then it's just passed as regular directory name. And at this point, it's possible to use Remapping constructs to access random remote locations while the client assumes to download from a subdirectory of apt-cacher-ng (as http server). This is configured by simply using /some/directory/string/ instead of URLs in the <code>Merging</code> list to let your clients download from http://acngserver/some/directory/string/... paths.
</p>
<p>
If multiple Remap- lines for the same <code>Repository</code> are specified, the contents of both URL lists are merged.
</p>
<p>
On some restricted networks, it may be needed to enforce the use of predefined mirrors. If the <code>ForceManaged</code> option is set, only requests to URL matched in some Remap-... config is allowed.
</p>
<p>
Sometimes, it may be needed to execute a system command before connection to certain machines is established. This is possible by associating commands with a repository declaration, i.e. by storing a file named like <code>repositoryname</code>.hooks in the main configuration directory. It can contain PreUp, Down and DownTimeout settings. PreUp/Down are executed by the system shell and it's up to the administrator to make sure that no malicious code is contained there and that the execution of these commands does not cause significant delays for other apt-cacher-ng users. See package documentation for an exemplary hooks file.
</p>
<p>
If the Redirection part contains multiple URLs, the server prefers to use them in the order of appearance. On success, the first target is used all the time, and so this should be the preferred mirror (note: "success" means getting a started download or a non-critical failure in this context. A "404 File not found" status is not considered critical since client's apt can expect and use it to check the existence of remote files and then change its own behaviour accordingly).
</p>
<p>
And finally, there is an optional third field in the <code>Remap</code> directives which can contain extra flags to modify downloading behavior in the scope of that particular cache repository.
</p>
<ul><li>
<code>keyfile=...</code> The meaning of this setting is: if any real download error (status code 400 and higher) happens on a file which path ends with the specified string then the target server is blacklisted (considered faulty) immediately and this download (and subsequent ones requested by this client connection) are retried from other servers (see <code>TargetURLs</code> description above). Can be used multiple times to define a list. See below for documented example.
</li>
<li>
<code>deltasrc=URL</code> Configures the base URL used to download .debdelta files. The path hierarchy below this URL should correspond to the source URLs and file paths in the cache. Only one URL can be specified at the moment. It is used for explicit mirroring operations, see <a href="howtos.html#mirroring">section 8.15</a> for details.
</li>
<li>
<code>proxy=proxyspec</code> Configures an alternative proxy URL which overrides the global proxy setting in the context of this repository. Can be set empty to disable proxy usage.
</li>
</ul>
<p>
Config example:
</p>
<pre><code>Remap-debrep: file:deb_mirror*.gz ; file:backends_debian ;
keyfile=Release keyfile=.deb
</code></pre>
<p>
If the first mirror from backends_debian goes wild and returns 404 responses for everything then the next candidate will be used. However, while this feature can improve redundancy for certain installations it needs to be used with care! Some file types are allowed to be missing and apt interprets their absence to change its behavior as needed. keyfile= should only match files which have an essential role and which disappearance is undoubtful indication of a broken server.
</p>
<hr><address>Comments to <a href='mailto:blade@debian.org'>blade@debian.org</a>
<br>
[Eduard Bloch, Sun, 19 Apr 2015 10:25:49 +0200]</address></body>
</html>
|