librefetch(8) -- downloads or creates a liberated source tarball
================================================================

## SYNOPSIS

`librefetch` [<OPTIONS>] <SOURCE-URL> [<OUTPUT-FILE>]<br>
`librefetch` `-`[`g`|`S`|`M`|`h`]

## DESCRIPTION

`librefetch` is a program to streamline creation of custom source
tarballs for `PKGBUILD(5)` files.

If a URL mentioned in the <source> array in a `PKGUILD` is in a
location that Parabola uploads "custom" source tarballs to (or
configured locations), and no file is at that URL, librefetch will
automatically create it for you.

This works because a post-install script for the package configures
`librefetch` as the download agent for `https://` URLs in
`makepkg.conf`; allowing it to jump in and create a file if need be.
Because of this, it is almost never necessary to call `librefetch`
manually.

The post-install script also configures `librefetch` as the download
agent for `libre://` URLs, for compatability with PKGBUILDs that used
a previous version of librefetch.

There are 5 modes:

 * `download`: Download the tarball from the configured mirror.
 * `create`: Create the tarball from a `PKGBUILD`/`SRCBUILD`.
 * `checksums`: Generate integrity checks for source files.
 * `srcbuild`: Print the effective build script.
 * `makepkg`: Print the effective makepkg script.
 * `help`: Print `librefetch` usage information.

The normal mode of operation is `download` mode.  If `download` mode
fails, it may choose to try `create` mode.

## OPTIONS

 * `-C`: Force `create` mode (don't download)
 * `-D`: Force `download` mode (don't create)
 * `-p` <file>: Use an alternate build script for `create` mode
   (instead of `PKGBUILD`).  If an `SRCBUILD` file exists in the same
   directory, it is used instead.
 * `-g` | `--geninteg`: Use `checksums` mode: Generate integrity
   checks for source files.
 * `-S` | `--srcbuild`: Use `srcbuild` mode: print the effective build
   script.
 * `-M` | `--makepkg`: Use `makepkg` mode: print the effective makepkg
   script.
 * `-h` | `--help`: Use `help` mode: Show useage information.

Other options, if they are documented in `makepkg -h`, are passed to
the modified copy of makepkg created during `create` mode.

## DOWNLOAD MODE

If <SOURCE-URL> begins with the string `libre://`, it is replaced with
the first value in <MIRRORS>, as configured in `librefetch.conf(5)`;
this is for compatability with PKGBUILDs that used a previous version
of librefetch.

It uses <DOWNLOADER>, as configured in `librefetch.conf` to attempt to
download the source tarball from that URL.  If that fails, and
following conditions are met, it proceeds to `create` mode:

 * The `-D` flag has not been specified to inhibit `create` mode.
 * The <SOURCE-URL> begins with one of the values in <MIRRORS>.

The latter requirement allows librefetch to be used as a generic
HTTP(S) download agent, that can automatically create files from
whitelisted locations.

## CREATE MODE

The principle of `create` mode is that a special `PKGBUILD(5)`, called
`SRCBUILD(5)`, that installs source files to <$pkgdir>, and the
resulting "package" is then used as a source tarball.  The `SRCBUILD`
exists in the same directory as the `PKGBUILD`.  It can be created
manually, or generated on-the-fly from the `PKGBUILD`.  Extra steps
are taken to ensure that as long as the same directory contents go in,
an identical tarball will come out--the checksum of the file should
not change based on when it is built or who builds it.

The `SRCBUILD` is either created, or sanitized if it already exists,
then fed to a modified version of `makepkg(8)`.

The purpose of the modified `makepkg` is so that the resulting tarball
doesn't contain package metadata, doesn't end with a .pkg file
extension, and always produces an identicle tarball.

When this documentation speaks of a file being modified, it is a
temporary copy of the file that is modified, your original file will
remain intact.

## SRCBUILD GENERATION

As explained in the `CREATE MODE` section, in `create` mode, this
program generates an `SRCBUILD` file.  For debugging purposes, this
file can be printed instead of executed with `srcbuild` mode.

### PRE-EXISTING SRCBUILD

The use of `SRCBUILD` files pre-dates `librefetch`.  By convention,
they set <PKGDEST> and <PKGEXT> in `package()` in order to modify the
behavior of `makepkg`.  Because a modified version of `makepkg` is
used, this interferes with the correct behavior.  To compensate for
this, lines containing "`PKGDEST=`" or "`PKGEXT=`" are deleted from
the `SRCBUILD`.

The general idea is that `build()` makes any modifications to the
source, then `package()` copies it from <$srcdir> to <$pkgdir>.

### SRCBUILD FROM PKGBUILD

Possibly more elegant than having a separate `SRCBUILD` file is having
an `mksource()` function in the main `PKGBUILD`.  This results in less
boilerplate and fewer files to edit.

Note that this only happens if a file named `SRCBUILD` doesn't already
exist; when migrating a package from a manually created `SRCBUILD` to
this method, the `SRCBUILD` must be deleted (or renamed) for this to
work.

The dynamically created `SRCBUILD` is created by copying `PKGBUILD` to
a temorary file, then re-setting variables and re-defining functions.
Following is a table of the translations.

    Variables
      source       = mksource
      noextract    = mknoextract
      *sums        = mk*sums (md5, sha1, sha256, sha384, sha512)
      depends      = <empty>
      checkdepends = <empty>
      makedepends  = mkdepends
    Functions
      prepare() { :; }
      build()   { mksource; }
      check()   { :; }
      package() { cp -a "$srcdir"/*/ "$pkgdir/"; }

The `mksource()` function does not need to be defined.  If it isn't
defined, then no transformations will be made to the source between it
being extracted to <$srcdir> and copied to <$pkgdir>.

In summary:

 * Set <mksource=()> and <mkmd5sums=()> to act as <source=()> and
   <md5sums=()>, respectively.
 * Declare a `mksource()` function to make modifications to the
   source, if nescessary.

Other changes:

 * <pkgname> is set to <pkgbase>, or the first element of the
   <pkgname> array (the effect is that split packaging is turnes
   off).
 * <options=()> is set have `makepkg` avoid making changes to
   <$pkgdir>.  The exact change is:

      options=(!strip docs libtool staticlibs emptydirs !zipman purge !upx)

 * <PURGE_TARGETS=()> has vcs directories added to it:

      PURGE_TARGETS=(.bzr/ .cvs/ .git/ .hg/ .svn/ .makepkg/)

### MAKEPKG MODIFICATIONS

The following modifications are made to makepkg:

 * Allow us to manipulate the output file (<$pkg_file>)
 * Do not include metadata in the output file (<${comp_files[@]}>)
 * Force 'ustar' tar format, don't allow it to upgrade to 'pax' to
   store extended file attributes.
 * Don't symlink the resulting file into the current directory.
 * <PURGE_TARGETS> interprets an item as a directory if it ends with a
   slash ("/").
 * Timestamps in <$pkgdir> are reset to "1990-01-01 0:0:0 +0", so that
   the resulting tarball will be the same, regardless of when it was
   created.
 * Sort the files included in the tarball; normally the order of files
   in a tarball is essentially random (even if it tends to be the same
   when re-created on the same machine).
 * append `-libre` to <$srcdir>
 * append `-libre` to <$pkgbasedir> (which becomes <$pkgdir>)
 * Don't check if the package has already been built.

For debugging purposes, this modified makepkg can be printed instead
of executed with `makepkg` mode.  Before it is run in create mode,
`PKGEXT`, `PKGDEST`, and `pkg_file` are set as environment variables.

## CONFIGURATION

See `librefetch.conf(5)` for details on configuring librefetch using
the `librefetch.conf` file.

## SEE ALSO

librefetch.conf(5), makepkg(8), PKGBUILD(5), SRCBUILD(5)