librefetch(8) -- downloads or creates a liberated source tarball
================================================================

## SYNOPSIS

`librefetch` [<OPTIONS>] <SOURCE-URL> [<OUTPUT-FILE>]<br>
`librefetch` `-`[`g`|`P`|`h`]

## DESCRIPTION

`librefetch` is a program to streamline creation of custom source
tarballs for `PKGBUILD(5)` files.

If a URL mentioned in the `source` array in a `PKGUILD` is in a
location that Parabola uploads "custom" source tarballs (or configured
locations), and no file is at that URL, librefetch will automatically
create it for you.

This works because a post-install script for the package configures
`librefetch` as the download agent for `https://` URLs in
`makepkg.conf`; allowing it to jump in and create a file if need be.
Because of this, it is almost never necessary to call `librefetch`
manually.

The post-install script also configures `librefetch` as the download
agent for `libre://` URLs, for compatability with PKGBUILDs that used
a previous version of librefetch.

There are 5 modes:

 * `download`: Download the tarball from the configured mirror.
 * `create`: Create the tarball from a `PKGBUILD`/`SRCBUILD`.
 * `checksums`: Generate integrity checks for source files.
 * `print`: Print the effective build script.
 * `help`: Print `librefetch` usage information.

The normal mode of operation is `download` mode.  If `download` mode
fails, it may choose to try `create` mode.

## OPTIONS

 * `-C`: Force `create` mode (don't download)
 * `-D`: Force `download` mode (don't create)
 * `-p` <file>: Use an alternate build script for `create` mode
   (instead of `PKGBUILD`).  If an `SRCBUILD` file exists in the same
   directory, it is used instead.
 * `-g` | `--geninteg`: Use `checksums` mode: Generate integrity
   checks for source files.
 * `-P` | `--print`: Use `print` mode: print the effective build script.
 * `-h` | `--help`: Use `help` mode: Show useage information.

## DOWNLOAD MODE

If <SOURCE-URL> begins with the string `libre://`, it is replaced with
the first value in `MIRRORS`, as configured in `librefetch.conf(5)`;
this is for compatability with PKGBUILDs that used a previous version
of librefetch.

It uses `DOWNLOADER`, as configured in `librefetch.conf` to attempt to
download the source tarball from that URL.  If that fails, and
following conditions are met, it proceeds to `create` mode:

 * The `-D` flag has not been specified to inhibit `create` mode.
 * The `<source-url>` begins with one of the values in `MIRRORS`.

The latter requirement allows librefetch to be used as a generic
HTTP(S) download agent, that can automatically create files from
whitelisted locations.

## CREATE MODE

The principle of `create` mode is that a special `PKGBUILD(5)`, called
a `SRCBUILD(5)`, installs source files to `$pkgdir`, and the resulting
"package" is then used as a source tarball.  The `SRCBUILD` exists in
the same directory as the `PKGBUILD`.  It can be created manually, or
generated on-the-fly from the `PKGBUILD`.  Extra steps are taken to
ensure that as long as the same directory contents go in, an identical
tarball will come out--the checksum of the file should not change
based on when it is built or who builds it.

The `SRCBUILD` is either created, or sanitized if it already exists,
then fed to a modified version of `makepkg(8)`.

The purpose of the modified `makepkg` is so that the resulting tarball
doesn't contain package metadata, doesn't end with a .pkg file
extension, and always produces an identicle tarball.

When this documentation speaks of a file being modified, it is a
temporary copy of the file that is modified, your original file will
remain intact.

## SRCBUILD GENERATION

As explained in the `CREATE MODE` section, in `create` mode, this
program generates an `SRCBUILD` file.  For debugging purposes, this
file can be printed instead of executed with `print` mode.

### PRE-EXISTING SRCBUILD

The use of `SRCBUILD` files pre-dates `librefetch`.  By convention,
they set `PKGDEST` and `PKGEXT` in `package()` in order to modify the
behavior of `makepkg`.  Because a modified version of `makepkg` is
used, this interferes with the correct behavior.  To compensate for
this, lines containing "`PKGDEST=`" or "`PKGEXT=`" are deleted from
the `SRCBUILD`.

The general idea is that `build()` makes any modifications to the
source, then `package()` copies it from `$srcdir` to `$pkgdir`.

### SRCBUILD FROM PKGBUILD

Possibly more elegant than having a separate `SRCBUILD` file is having
an `mksource()` function in the main `PKGBUILD`.  This results in less
boilerplate and fewer files to edit.

Note that this only happens if a file named `SRCBUILD` doesn't already
exist; when migrating a package from a manually created `SRCBUILD` to
this method, the `SRCBUILD` must be deleted (or renamed) for this to
work.

The dynamically created `SRCBUILD` is created by copying `PKGBUILD` to
a temorary file, then re-setting variables and re-defining functions.
Following is a table of the translations.

    Variables
      source       = mksource
      noextract    = mknoextract
      *sums        = mk*sums (md5, sha1, sha256, sha384, sha512)
      depends      = <empty>
      checkdepends = <empty>
      makedepends  = mkdepends
    Functions
      prepare() { :; }
      build() { mksource; }
      check() { :; }
      package() { cp -a "$srcdir"/*/ "$pkgdir/"; }

The `mksource()` function does not need to be defined.  If it isn't
defined, then no transformations will be made to the source between it
being extracted to `$srcdir` and copied to `$pkgdir`.

In summary:

 * Set `mksource=()` and `mkmd5sums=(`) to act as `source=(`) and
   `md5sums=()`
 * Declare a `mksource()` function to make modifications to the
   source, if nescessary.

Other changes:

 * `pkgname` is set to `pkgbase`, or the first element of the
   `pkgname` array.
 * `options=()` is set have `makepkg` avoid making changes to
   `$pkgdir`.  The exact change is:

      options=(!strip docs libtool staticlibs emptydirs !zipman purge !upx)

 * `PURGE_TARGETS=()` has vcs directories added to it:

      PURGE_TARGETS=(.bzr/ .cvs/ .git/ .hg/ .svn/ .makepkg/)

### MAKEPKG MODIFICATIONS

The following modifications are made to makepkg:

 * Allow us to manipulate the output file (`$pkg_file`)
 * Do not include metadata in the output file (`${comp_files[@]}`)
 * Force 'ustar' tar format, don't allow it to upgrade to 'pax' to
   store extended file attributes.
 * Don't symlink the resulting file into the current directory.
 * `PURGE_TARGETS` interprets an item as a directory if it ends with a
   slash ("/").
 * Timestamps in `$pkgdir` are reset to "1990-01-01 0:0:0 +0", so that
   the resulting tarball will be the same, regardless of when it was
   created.
 * Sort the files included in the tarball; normally the order of files
   in a tarball is essentially random (even if it tends to be the same
   when re-created on the same machine).
 * append `-libre` to `$srcdir`
 * append `-libre` to `$pkgbasedir` (which becomes `$pkgdir`)
 * Don't check if the package has already been built.

## CONFIGURATION

See `librefetch.conf(5)` for details on configuring librefetch using
the `librefetch.conf` file.

## SEE ALSO

librefetch.conf(5), makepkg(8), PKGBUILD(5), SRCBUILD(5)