[Fixes #14381] Relax Remote Service URL uniqueness - #14404
Open
etj wants to merge 14 commits into
Open
Conversation
Contributor
Author
|
Waiting for #14328 to be merged. |
…ath in test_get_service_handler_wms
…andler Leftover from the earlier manual "Align to master" commit; harmless but redundant.
_get_service_handler (views.py) and GeoNodeServiceHandler.parsed_service
(wms.py) build the request auth via get_request_auth(), which returns a
HashableAuthBase-wrapped requests.auth object, but only passed auth= to
get_service_handler, not auth_config=. Since HashableAuthBase exposes no
`.username` attribute, _build_auth_cache_fingerprint fell through to a
fingerprint that was constant for every HashableAuthBase instance
("auth:HashableAuthBase"), so changing a service's stored credentials
would not bust the cached service handler until it naturally expired.
Fix both call sites to also pass auth_config=service.auth_config,
consistent with every other get_service_handler/get_service_cache_key
call site. Also harden _build_auth_cache_fingerprint itself to unwrap
HashableAuthBase and fingerprint the full underlying credential dict,
so the auth= path is correct on its own rather than relying solely on
callers always supplying auth_config too.
parsed_service calls self.ows_endpoint() (a real HTTP request) before get_cleaned_url_params, which wasn't mocked, causing a real DNS lookup against the phony test hostname. Mock ows_endpoint too.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
geonode/services/serviceprocessors/init.py:61
logger.exception(msg=...)is an invalid call signature for the stdlib logger (the message must be the first positional argument). In the exception handler this will raise aTypeErrorand can mask the original parsing error. Log the message positionally (optionally with%sformatting) and keep the original exception context.
except Exception as e:
logger.exception(e)
logger.exception(msg=f"Could not parse service {base_url}")
raise
mattiagiupponi
approved these changes
Jul 17, 2026
mattiagiupponi
requested changes
Aug 10, 2026
Contributor
There was a problem hiding this comment.
@etj i tried to use it, but when i try to import a resource from another service with the same URL, the following error is raised
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/core/geonode/geonode/harvesting/tasks.py", line 348, in _harvest_resource
worker.update_geonode_resource(
File "/opt/core/geonode/geonode/harvesting/harvesters/base.py", line 223, in update_geonode_resource
geonode_resource = self._create_new_geonode_resource(geonode_resource_type, defaults)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/core/geonode/geonode/harvesting/harvesters/base.py", line 253, in _create_new_geonode_resource
geonode_resource = resource_manager_registry.get_for_model(geonode_resource_type).create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/core/geonode/geonode/resource/manager.py", line 353, in create
_resource, _created = resource_type.objects.get_or_create(uuid=uuid, defaults=resource_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/query.py", line 955, in get_or_create
return self.create(**params), True
^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/query.py", line 665, in create
obj.save(force_insert=True, using=self.db)
File "/opt/core/geonode/geonode/base/models.py", line 1090, in save
super().save(*args, **kwargs)
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/polymorphic/models.py", line 86, in save
return super().save(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/base.py", line 902, in save
self.save_base(
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
updated = self._save_table(
^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/base.py", line 1169, in _save_table
results = self._do_insert(
^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/base.py", line 1210, in _do_insert
return manager._insert(
^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/query.py", line 1873, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
cursor.execute(sql, params)
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/backends/utils.py", line 122, in execute
return super().execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/mattia/.virtualenvs/gn5/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.IntegrityError: duplicate key value violates unique constraint "layers_dataset_store_workspace_name_0b4228a1_uniq"
DETAIL: Key (store, workspace, name)=(httpswmsigngobargeoserveriderabahrawms, remoteWorkspace, bahra) already exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the identity model of remote services from endpoint URL uniqueness to service record uniqueness.
Before:
base_url) implicitly identified one service.After:
base_urlis no longer a unique identifier.Serviceobjects can intentionally reference the same endpoint (e.g., different owners/use-cases/credentials).What this PR changes
Service.base_url.Servicerecords pointing to the same remote endpoint.base_urlvalues.Service.base_urlfield.geoserver_pre_save_maplayerfallback strict on ambiguous dataset matches (logs + raises onMultipleObjectsReturned).ServiceHandlerCacheclass (serviceprocessors/cache.py), mirroring the existingServiceTypeRegistrypattern, instead of loose module-level functions and a duplicated cache handle inviews.py.GeoNodeServiceHandler.parsed_service(predates this PR) that read auth/type/id off a variable that is actually just the rawservice=query-string value, not a Service model -- this would raiseAttributeErrorthe moment the property ran (e.g. registering or probing a GeoNode-type remote service). Auth now resolves the same wayWmsServiceHandler.parsed_servicedoes.Solved
Note: this PR may conflict with #14328Checklist
For all pull requests:
The following are required only for core and extension modules (they are welcomed, but not required, for contrib modules):
Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.