summaryrefslogtreecommitdiff
path: root/cron-jobs
AgeCommit message (Collapse)Author
2010-08-10Use more consitent naming for package poolPierre Schmitz
There are no longer architecture-specific subdirs and the structure was switch to this: ftp └── pool ├── community └── packages packages contains all packages from core, extra and testing; this naming is in sync with the svn repo naming: svn-packages and svn-community
2010-08-08Use db-functions in create-filelists and sourceballsPierre Schmitz
2010-08-08Cleanup cron-jobs/adjust-permissionsPierre Schmitz
2010-08-08Move common function to db-functionsPierre Schmitz
db-functions now sets an individual $WORKDIR and implements trap functinos that remove locks on exit or error. There are new functions to lock and unlock the running script. misc-scripts/ftpdir-cleanup was renamed to ftpdir-cleanup-repo as the cron-job had the same name. Script names have to be unique when using db-functions.
2010-08-08fix cleaning of any packages and ignore empty reposPierre Schmitz
2010-06-28sourceballs: Added community and community-testing supportEric Bélanger
Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com> Signed-off-by: Pierre Schmitz <pierre@archlinux.de>
2010-06-20adjust permissions for package poolPierre Schmitz
2010-06-20ftpdir-cleanup: only search for linked packages in actual reposPierre Schmitz
2010-06-20Replace PKGPOOL_DIR by a function callPierre Schmitz
The package pool dir depends on the host; so we solve this similar to get_repos_for_host() Note: There is no "os" subdir for the package pool.
2010-05-03Add PKGPOOL_DIR config variableAaron Griffin
This is a relative var, descendent from FTP_BASE. We do this because we also need a relative path to the package pool dir
2010-04-28ftpdir-cleanup: use 'find' for symlink detectionAaron Griffin
This gets rid of a bash loop for link detection
2010-04-28ftpdir-cleanup: Fix cleanup of package poolAaron Griffin
2010-04-28Change ftpdir-cleanup scripts for package poolingAaron Griffin
* The per repo script now simply removes files as they should all be symlinks at this time * The cron job script will do the extra job of checking the global pool. If there are no more symlinks, move the pooled package Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2010-03-31don't publish *.old filesPierre Schmitz
2010-03-13moving sourceballs-cleanupEric Bélanger
2010-03-13sourceballs: Make cleanup more efficientEric Bélanger
Moved all cleanup related code in sourceballs-cleanup script and moved, now common, functions in db-functions. The cleanup script is now ran after all the new sourceballs have been fetched. Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
2010-02-27create-filelists: include desc/depends entriesDan McGee
Make the files DB include everything the original packages DB includes instead of just being 'files' entries. This will allow tools to do more with these generated files and they can be used as a drop-in replacement for a regular package database. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-27create-filelists: rework the package loop completelyDan McGee
Instead of wasting time extracting .PKGINFO twice from every single package in the repos, use the package DB to eliminate most of the heavy lifting. This way we only need to worry about looking at the packages that actually have changed since the last time we built the package database. This should give a noticeable performance increase to this job in addition to reducing IO load and unnecessary reading of every package file. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-27create-filelists: s/REPO_DB_FILE/FILES_DB_FILE/gDan McGee
This will set up changes soon to come where we actually use the real repos DB file so I don't want variable name confusion. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-02-27create-filelists: general cleanupsDan McGee
* Specify lock name once * Use new script name everywhere * Clean up tabs/spaces and add a modeline. This isn't necessarily the one we wanted to standardize on, but I picked the one the entire file is written to at the moment. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-10-31create-filelists: Add pkgver-pkgrel to directory nameThomas Bächler
In the past, pkgname-pkgver-pkgrel was extracted from the filename and sadly named "pkgname". Restore this behaviour
2009-10-31create-filelists: fix another typoThomas Bächler
2009-10-31create-filelists: add community-testingThomas Bächler
2009-10-31Fix a typo introduced in 38c7241ad2528b29a65a0b81b980e76d1046e94eThomas Bächler
2009-10-29create-filelists: Call getpkgname for realAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-10-29Cleanup create-filelistsAaron Griffin
* Quote absolutely everything * Move directory slashes around, for readability * Move a cd to the top of the loop Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-10-29Rename createFileLists to be more like the othersAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-10-26Added new cronjob script to clean up unneeded sourceballsEric Bélanger
This script is intended to be a weekly cron-job. It supplements the current sourceballs cleanup in make-sourceball that only removes the old sourceballs. This script removes the sourceballs of packages that were either removed completely from the repo or moved to a splitted package. It also checks the license of the packages which have a sourceball to see, in case of a license change, if the hosting of the sources is still necessary. Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-25sourceballs: skip based on pkgbase failuresAaron Griffin
If one portion of a split package fails, assume that all packages from the same $pkgbase are going to fail as well and skip them for this run. Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-24Fixed test for existence of split package sourcesAaron Griffin
This patch also removes the unused CARCH variable. Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com> [Aaron: Broke srcpkgbase logic into srcpkgname/srcpkgbase to make it more readable] Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-21Add a safety check for existence in pkg loopAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-21Set/restore umask in the sourceballs cron jobAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-21Temporarily disable community sourceball generationAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-21Merge branch 'master' of file:///srv/projects/git/dbscriptsAaron Griffin
2009-09-21check_packages : fix a bug when filling provisionsXavier Chantry
When we had a duplicate package, we would replace the old ref by the new ref. But the provisions dict kept both the old and new ref. This was not good at all. Now, we just keep the old ref, and we only fill provisions after all packages have been parsed. This should be much more sane. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18integrity-check: Rename $3 -> $2Aaron Griffin
OMFG such a simple script and I keep missing crap Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18integrity-check: Forgot to decrement the arg countAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18integrity-check: Loop over archesAaron Griffin
THis simplifies the command line and we do both anyway Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18check_archlinux: absroot -> absrootsAaron Griffin
Allow us to use the --abs-root flag. He he Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18Bump check_archlinux module for libalpm.so.4Aaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18parse_pkgbuilds.sh : fix pkgbase handlingXavier Chantry
add the default value for pkgbase also rework slightly how variables are handled (better distinction between splitpkg variables and the rest) Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18improve the way parse_pkgbuilds.sh is calledXavier Chantry
1) check the return value. before a failed call to parse_pkgbuilds.sh was completely silent.. not good :) 2) call realpath on check_packages.py before determining the directory name, to figure out where parse_pkgbuilds.sh is. This allows to symlink check_packages.py and use the symlink without moving parse_pkgbuilds.sh . for example : /foo/bar/ contains check_packages.py and parse_pkgbuilds.sh ln -s /foo/bar/check_packages.py /bin/check_package dirname(/bin/check_package) returns /bin/ so not good dirname(realpath(/bin/check_package)) returns /foo/bar/ so good Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18parse_pkgbuilds : simple performance tweakXavier Chantry
this simple tweak gives a nice perf boost : from 10s to 7s to parse extra repo. indeed calling basename caused a fork to happen for the thousand of files being considered Now the major bottleneck is parsing split pkgbuilds, but I am afraid there is no magical solution for that :( Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18improve compute_dep algorithmXavier Chantry
The compute_dep function I wrote for the more informative hierarchy output was very inefficient. It's much better now (10s -> 0.5s) on my box, and I get exactly the same results :) Now the big majority of the time is again spent on parsing pkgbuilds. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18add the repo/ info to most results, and sort themXavier Chantry
Add repo information to most results by using the repo/pkgname syntax (instead of only pkgname) Sort all results list (with the point above, this sorts the results by repo :)) Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18Add db comparison to integrity checkHenning Garus
Compare the abs tree with the repo dbs to check if we have a PKGBUILD for each package in the dbs and vice versa. Signed-off-by: Henning Garus <henning.garus@gmail.com> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-18check_packages.py: Allow multiple abs-treesHenning Garus
By parsing multiple abs trees we can add any when parsing the other trees, checking any standalone doesn't make much sense. Signed-off-by: Henning Garus <henning.garus@gmail.com> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>
2009-09-13Maybe I should make the script really adjust permissionsDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-13Add new get_repos_for_host() functionDan McGee
And use it. This allows us to have server-specific behavior in our scripts without further patching, and it also allows us to simplify some of our scripts a fair amount. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-13Be more careful with config file inclusionDan McGee
We can't really do the basedir magic from db-functions as it is just being sourced and is not the currently executing script. Although a bit repetitive, it is a lot safer to just include the config file everywhere. Noticed this when trying to run the latest available scripts on sigurd. Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3-2-g168b (git 2.25.1) at 2024-10-30 10:02:35 +0000