Bug #31254
closed
"Package matching query does not exist." error when syncing two repositories, that contain identical packages
Description
When syncing the Oracle Linux repositories "Oracle Linux 7 (x86_64) Latest" (http://yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64) and "Oracle Linux 7 (x86_64) Optional Latest" (http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64), the synchronization of the repo, which is synced after the other one fails with the error "Package matching query does not exist".
When looking at the pulp worker logs, I can see, that the database throws a duplicate key error, because some packages are present in both repositories:
Examples:
c-ares-devel-1.10.0-3.el7.i686.rpm
c-ares-devel-1.10.0-3.el7.x86_64.rpm
cdparanoia-10.2-17.el7.x86_64.rpm
I found this behaviour in both Katello 3.16 and in Katello 3.17RC2.
As I don't have any possibility to change the foreign repositories, it would be nice to have either a possibility to filter out the packages like in Content Views, so that they are not synced for specific repositories, or if the duplicate would be detected by an error handling and the existing package is also linked to the other repository (no duplicate in the file system, but an additional reference in the database).
I attached a PDF with the error in the Foreman interface, the pulp worker log with the error and the corresponding entry from the database.
Kind regards
Carsten
Files
Updated by Ian Ballou over 3 years ago
- Target version set to Katello 3.16.2
- Triaged changed from No to Yes
TODO: We should investigate if this is strictly a Pulp 3 bug.
Updated by Carsten Menzel over 3 years ago
Good morning,
yesterday I did some further tests to see, why the problem occurs.
It seems to me, that the problem occurs, if a product with multiple large repositories is generated and is initially synced in parallel (complete product sync).
##########
Commands to create an example product, which contains the two repositories that contain some identical packages ("OL 7 - Latest" and "OL 7 - Optional Latest"):
hammer product create --organization "<organization name>" --name "Oracle Linux"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 6 - Latest" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL6/latest/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 6 - Addons" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL6/addons/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 7 - Latest" --content-type "yum" --url "https://public-yum.oracle.com/repo/OracleLinux/OL7/latest/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 7 - Optional Latest" --content-type "yum" --url "https://public-yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 7 - Addons" --content-type "yum" --url "https://public-yum.oracle.com/repo/OracleLinux/OL7/addons/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 7 - Software Collection" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL7/SoftwareCollections/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 8 - AppStream Latest" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 8 - Addons" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL8/addons/x86_64/"
hammer repository create --organization "<organization name>" --product "Oracle Linux" --name "OL 8 - BaseOS Latest" --content-type "yum" --url "https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/"
###########
Sync the complete product, so that all repositories are synced at the same time.
This seems to lead to the error situation.
Perhaps this is a timing issue when a proxy is used (which slows down the downloads a bit).
Yesterday I deleted in my system both repositories ("OL 7 - Latest" and "OL 7 - Optional Latest"), removed all orphaned content and recreated the two repositories.
After that, I synced the Repos one after the other (the first one yesterday in the evening, the second one today in the morning)
In this case, the error did not occur anymore.
Kind Regards
Carsten
Updated by Carsten Menzel over 3 years ago
As this issue was a temporary problem and could be caused by a timing issue during the initial sync, I don't think, that this is a general bug anymore.
As I now know how to avoid the problem, you can close the bug (either as resolved or as rejected).
Regards
Carsten
Updated by Justin Sherrill over 3 years ago
I think this is a pulp3 bug still worth investigating. I filed: https://pulp.plan.io/issues/7828
Updated by Ian Ballou over 3 years ago
- Target version changed from Katello 3.16.2 to 1341
Updated by Ian Ballou over 3 years ago
- Target version changed from 1341 to Katello 3.17.2
Updated by Chris Roberts over 3 years ago
- Target version changed from Katello 3.17.2 to Katello 3.17.3
Updated by Jonathon Turel about 3 years ago
- Target version changed from Katello 3.17.3 to Katello 4.1.0
Updated by Jonathon Turel almost 3 years ago
- Status changed from New to Closed
- Target version changed from Katello 4.1.0 to Katello Recycle Bin
Closing this out since the root of the issue is on the Pulp side. However we will raise this with pulp devs and slot the fix into a future Katello release. Thanks for filing :)