Age | Commit message (Collapse) | Author |
|
There are no longer architecture-specific subdirs and the structure was switch to this:
ftp
└── pool
├── community
└── packages
packages contains all packages from core, extra and testing; this naming is in sync with the svn repo naming:
svn-packages and svn-community
|
|
|
|
|
|
db-functions now sets an individual $WORKDIR and implements trap functinos that
remove locks on exit or error. There are new functions to lock and unlock the running script.
misc-scripts/ftpdir-cleanup was renamed to ftpdir-cleanup-repo as the cron-job had the same name.
Script names have to be unique when using db-functions.
|
|
|
|
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
|
|
|
|
|
|
The package pool dir depends on the host; so we solve this similar to
get_repos_for_host()
Note: There is no "os" subdir for the package pool.
|
|
This is a relative var, descendent from FTP_BASE. We do this because
we also need a relative path to the package pool dir
|
|
This gets rid of a bash loop for link detection
|
|
|
|
* The per repo script now simply removes files as they should all be
symlinks at this time
* The cron job script will do the extra job of checking the global
pool. If there are no more symlinks, move the pooled package
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
|
|
|
|
Moved all cleanup related code in sourceballs-cleanup script and moved,
now common, functions in db-functions. The cleanup script is now ran
after all the new sourceballs have been fetched.
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
|
|
Make the files DB include everything the original packages DB includes
instead of just being 'files' entries. This will allow tools to do more with
these generated files and they can be used as a drop-in replacement for a
regular package database.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Instead of wasting time extracting .PKGINFO twice from every single package
in the repos, use the package DB to eliminate most of the heavy lifting.
This way we only need to worry about looking at the packages that actually
have changed since the last time we built the package database.
This should give a noticeable performance increase to this job in addition to
reducing IO load and unnecessary reading of every package file.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will set up changes soon to come where we actually use the real repos
DB file so I don't want variable name confusion.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
* Specify lock name once
* Use new script name everywhere
* Clean up tabs/spaces and add a modeline. This isn't necessarily the one we
wanted to standardize on, but I picked the one the entire file is written
to at the moment.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
In the past, pkgname-pkgver-pkgrel was extracted from the filename and sadly named "pkgname". Restore this behaviour
|
|
|
|
|
|
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
* Quote absolutely everything
* Move directory slashes around, for readability
* Move a cd to the top of the loop
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
This script is intended to be a weekly cron-job. It supplements the current sourceballs cleanup in make-sourceball that only removes the old sourceballs. This script removes the sourceballs of packages that were either removed completely from the repo or moved to a splitted package. It also checks the license of the packages which have a sourceball to see, in case of a license change, if the hosting of the sources is still necessary.
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
If one portion of a split package fails, assume that
all packages from the same $pkgbase are going to fail
as well and skip them for this run.
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
This patch also removes the unused CARCH variable.
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
[Aaron: Broke srcpkgbase logic into srcpkgname/srcpkgbase
to make it more readable]
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
|
|
When we had a duplicate package, we would replace the old ref by the new
ref. But the provisions dict kept both the old and new ref. This was not
good at all.
Now, we just keep the old ref, and we only fill provisions after all
packages have been parsed. This should be much more sane.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
OMFG such a simple script and I keep missing crap
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
THis simplifies the command line and we do both anyway
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Allow us to use the --abs-root flag. He he
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
add the default value for pkgbase
also rework slightly how variables are handled (better distinction between
splitpkg variables and the rest)
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
1) check the return value. before a failed call to parse_pkgbuilds.sh was
completely silent.. not good :)
2) call realpath on check_packages.py before determining the directory name,
to figure out where parse_pkgbuilds.sh is. This allows to symlink
check_packages.py and use the symlink without moving parse_pkgbuilds.sh .
for example : /foo/bar/ contains check_packages.py and parse_pkgbuilds.sh
ln -s /foo/bar/check_packages.py /bin/check_package
dirname(/bin/check_package) returns /bin/ so not good
dirname(realpath(/bin/check_package)) returns /foo/bar/ so good
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
this simple tweak gives a nice perf boost : from 10s to 7s to parse extra
repo.
indeed calling basename caused a fork to happen for the thousand of files
being considered
Now the major bottleneck is parsing split pkgbuilds, but I am afraid there
is no magical solution for that :(
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
The compute_dep function I wrote for the more informative hierarchy output
was very inefficient. It's much better now (10s -> 0.5s) on my box, and I
get exactly the same results :)
Now the big majority of the time is again spent on parsing pkgbuilds.
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Add repo information to most results by using the repo/pkgname syntax
(instead of only pkgname)
Sort all results list (with the point above, this sorts the results by
repo :))
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Compare the abs tree with the repo dbs to check if we have a PKGBUILD for
each package in the dbs and vice versa.
Signed-off-by: Henning Garus <henning.garus@gmail.com>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
By parsing multiple abs trees we can add any when parsing the other trees,
checking any standalone doesn't make much sense.
Signed-off-by: Henning Garus <henning.garus@gmail.com>
Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
And use it. This allows us to have server-specific behavior in our scripts
without further patching, and it also allows us to simplify some of our
scripts a fair amount.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We can't really do the basedir magic from db-functions as it is just being
sourced and is not the currently executing script. Although a bit
repetitive, it is a lot safer to just include the config file everywhere.
Noticed this when trying to run the latest available scripts on sigurd.
Signed-off-by: Dan McGee <dan@archlinux.org>
|