Age | Commit message (Collapse) | Author |
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We were spending a lot of time getting the developer name for a given
key on this page, which involved sending one query per key ID. Use the
cache to our advantage here and save ourselves the "expensive" lookups.
This eliminates ~100 queries per page load.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
|
|
For something like gambas3 which has a makedepend on postgresql, we end
up getting every single split package listed in the required by list for
postgresql. This is a bit crazy and unnecessary, so slim it down a bit
when possible by using a slightly crazy groupby function and some smarts
in our get_requiredby function.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We've moved onto bytes only now.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Now that we have a BinaryField option in Django 1.6, put it to use.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will indicate how many signoffs are required for packages of the
given architecture. Set the default to 2 as it currently stands.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We were doing this for depends entries already, and we can do the same
for required by entries via some absolutely awesome SQL query magic and
injecting a few more things to our already crazy query.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Most of these were suggested by PyCharm, and include everything from
little syntax issues and other bad smells to dead or bad code.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We sometimes see some web bots and crawlers make HEAD requests to verify
existence of certain pages in the application. However, they are less
than kind as 20-50 requests might arrive at the same time, and package
search and details pages are some of the slowest rendering pages we have
due to the Django template engine.
Rather than waste time generating the content only to throw it away,
response as soon as we can with either a 404 or 200 response as
appropriate, omitting the 'Content-Length' header completely, which
seems to be acceptable by the HTTP spec.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The benefit of these storage operations might be outweighed by the cost,
especially given how infrequently these functions are called.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Seems the Django caching layer has changed quite a bit and is doing all
sorts of funky s**t at this point. Yay for errors!
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We started seeing this in production with memcached usage and the
upgrade to Django 1.5:
PicklingError: Can't pickle <class 'devel.models.UserProfile_allowed_repos'>: attribute lookup devel.models.UserProfile_allowed_repos failed
Without having time to investigate further, something changed, likely
due to the whole user profile/configurable user model shenanigans done
in Django 1.5. For now, simply don't cache this attribute to work around
the problem.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Rather than use the Django cache for this (aka memcached), just do it on
a per-object instantiation basis. This means no external services calls
except the first time to the database, which should be quite a bit
faster.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This reverts commit 20b64e42672d185821cc584dfa4b133ee259a144.
Django 1.5 fixed this issue and now parent objects are automatically
attached to their children when queries go through the related manager.
See "Caching of related model instances" in the release notes.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This doesn't do any super optimizations, but does run the very basic
cssmin and jsmin Python tools over the static resources we serve up.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
We aren't looking up users; we are looking up developer keys.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This introduces the new model to the package page so subkey signings
show up as attributed to the original developer. We also teach the
mismatched signatures report to recognize all keys and subkeys of a
given developer, cutting down on some of the bogus results.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The first of several small updates to use the new data we have
available.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The query refactor in commit 1b1b516bd removed a queryset I didn't
realize was getting used elsewhere in the function.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Let's just go with the Django database option for PostreSQL autocommit
mode instead.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For packages with particularly long lists of provides (e.g. perl), the
query was getting a bit out of control with the list of names passed in.
However, changing it to simply do a subquery resulted in some really
poor planning by PostgreSQL. Doing this as a custom 'WHERE' clause
utilizing the 'UNION ALL' SQL operator works very well.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This uses a new template tag to avoid repeating construction of the
necessary HTML element all over the place. The site should look exactly
as it did before, except now you don't have to download 20+ images to
see some pages.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This moves most migrations to the v2 format that have been presenting
some issues. One missing depends_on relationship has been added, and we
allow an index to not be dropped if it does not exist due to the
shittyness in sqlite3 actually keeping indexes across DDL on that table.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
The reason for this is documented in the middleware itself. Without
this, pgbouncer is of little use to us since it has to throw away every
connection we try to route through it because of unclean disconnects.
In theory, with the switch to using pgbouncer for all WSGI originating
connections and adding this middleware, we should see a notable decrease
in connection time to the database.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This will be used to eventually implement the UI side of FS#13441, but
to do that, we first need the data.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
This is the preferred and non-deprecated way of doing this in Django
1.4+.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Since we need these in the template for any details links, we might as
well pull them back from the database in one query rather than three.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Django doesn't attach the parent object to the child objects, even when
queried through the related manager. This causes up to 3 extra queries:
one to retrieve the package again, and one each for arch and repo
retrieval.
For a package like archboot, this drops the number of necessary queries
for the package page from 805 to 222 (yes, this is still too high) and
cuts the time spent waiting on the database from 505ms to 262ms.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Instead of having multiple methods, move this into our single 'created'
setter method. If the 'last_modified' property is present, we now update
it accordingly when saving any model with this signal attached.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
When running tests, we can find old migrations that didn't use datetime
objects with timezones attached.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Fix some that referenced non-existent attributes, and add the attribute
to other models.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
For package signatures, it turns out it is way cheaper to just parse the
signature again rather than going though all the decorator and
cache_function_key business. This speeds up the mismatched signatures
report significantly once this is removed.
For base_package, given that we only call it once from our package
details template, it makes little sense to cache the result.
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|
|
Signed-off-by: Dan McGee <dan@archlinux.org>
|