Actions
Bug #29360
closed
Too-large request sent to Pulp2to3MigrationClient during katello:pulp3_migration
Description
I hit this issue in a pretty specific situation. My environment had two file repos synced with this URL: http://quartet.usersys.redhat.com/pub/fake-repos/large_file/
I also had a content view with 3 versions, each having this large file repo.
I deleted every repo and content view version (trying to make orphaned content) and then tried katello:pulp3_migration. I hit this error:
[vagrant@centos7-katello-nightly-3 ~]$ sudo foreman-rake katello:pulp3_migration
rake aborted!
ForemanTasks::TaskError: Task b7e58204-6555-493b-aaa6-e1b46aaedba6: Dynflow::Errors::UnknownError[Pulp2to3MigrationClient::ApiError]: Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Mon, 16 Mar 2020 21:28:09 GMT", "server"=>"Apache", "content-type"=>"text/html; charset=UTF-8", "content-length"=>"163", "via"=>"1.1 centos7-katello-nightly-3.cannolo.example.com", "connection"=>"close"}
Response body: <html>
<head>
<title>Bad Request</title>
</head>
<body>
<h1><p>Bad Request</p></h1>
Request Line is too large (7970 > 4094)
</body>
</html>
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.0.pre.master/lib/katello/tasks/pulp3_migration.rake:13:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
I recreated the error in the Rails console like so:
irb(main):015:0> ::Katello::FileUnit.where(:migrated_pulp3_href => nil)
=> #<ActiveRecord::Relation [#<Katello::FileUnit id: 1, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/2c69ee03-e687-4313...", name: "35689", checksum: "9f59a0641b8a7848452719b1e90624f84d80be6310fb7118f9...", path: "35689", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 2, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/2ddf5ec4-6d37-454b...", name: "10440", checksum: "5b8850b8ef0f5addb47a772d75dcfcd370a9cbfeefa9d53a45...", path: "10440", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 3, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/5b5941a1-af0b-47ce...", name: "14221", checksum: "e892e084e3956eee21fd52d1daa2a67daf90bd77d80daa32ef...", path: "14221", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 4, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/7974bc79-9c0d-4467...", name: "55172", checksum: "52b920ea83fb66b5bc91ba6fe1c29bc6f8b21c65c0e35c5871...", path: "55172", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 5, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/61f2585b-b936-41cb...", name: "24571", checksum: "22c6580001d68bf813770a0027504cfe561aba11083bfa4c21...", path: "24571", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 6, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/9ce8e88a-aaa1-4218...", name: "25771", checksum: "e8e02ee843065c3166425f2242deee27c1e3697cd615506909...", path: "25771", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 7, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/7ae58e78-b10a-4a5d...", name: "64608", checksum: "49e19949f177cb981acf74bd554cac692d6291a78125c750af...", path: "64608", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 8, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:25", pulp_id: "/pulp/api/v3/content/file/files/9f79c4d6-6f5c-4069...", name: "50039", checksum: "8461a0d270b6db87153f813c9156530f285008848cfef5f674...", path: "50039", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 9, created_at: "2020-03-16 16:01:25", updated_at: "2020-03-16 16:01:26", pulp_id: "/pulp/api/v3/content/file/files/73cc04d8-a94f-4ef8...", name: "30182", checksum: "ae53b9771f94d4aa4760b6071213797456d027ed69dc8bd120...", path: "30182", migrated_pulp3_href: nil>, #<Katello::FileUnit id: 10, created_at: "2020-03-16 16:01:26", updated_at: "2020-03-16 16:01:26", pulp_id: "/pulp/api/v3/content/file/files/428d27e9-bcc6-471a...", name: "34288", checksum: "197d6e20273ba80188a988bd207ceb057c069c94447383411c...", path: "34288", migrated_pulp3_href: nil>, ...]>
irb(main):016:0> ::Katello::FileUnit.where(:migrated_pulp3_href => nil).count
=> 67985
irb(main):017:0> Pulp2to3MigrationClient::Pulp2contentApi.new(Pulp2to3MigrationClient::ApiClient.new(SmartProxy.first.pulp3_configuration(Pulp2to3MigrationClient::Configuration))).list
=> #<Pulp2to3MigrationClient::InlineResponse2001:0x000000000fa99020 @count=0, @results=[]>
irb(main):018:0> Pulp2to3MigrationClient::Pulp2contentApi.new(Pulp2to3MigrationClient::ApiClient.new(SmartProxy.first.pulp3_configuration(Pulp2to3MigrationClient::Configuration))).list(pulp2_id__in: ::Katello::FileUnit.where(:migrated_pulp3_href => nil).map { |unit| unit.pulp_id }.join(','))
Traceback (most recent call last):
2: from lib/tasks/console.rake:5:in `block in <top (required)>'
1: from (irb):18
Pulp2to3MigrationClient::ApiError (Error message: the server returns an error)
HTTP status code: 414
Response headers: {"date"=>"Mon, 16 Mar 2020 21:31:40 GMT", "server"=>"Apache", "content-length"=>"248", "connection"=>"close", "content-type"=>"text/html; charset=iso-8859-1"}
Response body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>414 Request-URI Too Long</title>
</head><body>
<h1>Request-URI Too Long</h1>
<p>The requested URL's length exceeds the capacity
limit for this server.<br />
</p>
</body></html>
The migration succeeded after deleting all file units and running both katello:delete_orphaned_content and katello:correct_repositories.
My environment did have some troubles, like for some reason not every file was synced correctly (you can see there isn't actually 70K files like there should be). Regardless it should be investigated.
Updated by 
Updated by 
Updated by
Actions