Repository navigation
Remote-resource detection is inconsistent across harvesters #14524
Description
Working on #14381 highlighted that store and workspace are meaningless for Datasets coming from remote services.
PR #14404 allowed store to be NULL.
workspace cannot follow yet, because it is used for unrelated checks inside GeoNode.
The blocker is that "is this dataset remote?" has three answers that disagree, depending on the harvester:
| harvester | sourcetype |
subtype |
remote_service |
is_remote_resource() |
|---|---|---|---|---|
| WMS | REMOTE | remote |
set | True |
| ArcGIS | REMOTE | remote |
set | True |
| GeoNode legacy | REMOTE | vector/raster |
set | False |
| GeoNode current | LOCAL | vector/raster |
never set | False |
Causes:
is_remote_resource()(resource/utils.py) keys offsubtype, but the GeoNode harvesters overwrite it with the remote's real subtype.sourcetype/subtypeare only set to REMOTE as a side effect of the remote not providing a lat/lon bbox (harvesting/harvesters/base.py:197-201).remote_service/remote_typenameare only assigned whensourcetype == REMOTE(resource/utils.py:261), so GeoNode-current-harvested datasets never get them.
Consequence: datasets harvested from a remote GeoNode look local, so they bypass the guards added in 4bc1154ba and the local GeoFence/GeoServer is called for them. get_dataset_workspace() returns the remote's workspace name (commonly the literal geonode) so set_permissions/remove_permissions can write and delete rules belonging to a local layer with the same name.
Repro: register a remote GeoNode service, harvest a dataset whose name matches an existing local layer, change its permissions, inspect GeoFence rules for geonode:<name>. Then delete the harvested dataset and observe the local layer's rules disappear.
Tasks in scope, in this order:
- Make remote detection single-sourced:
is_remote_resource()offsourcetype/remote_serviceinstead ofsubtype; stop derivingsourcetypefromlonlat_extent; setremote_service/remote_typenamefor all harvested datasets. - Guard the GeoFence/GeoServer paths accordingly (
geoserver/manager.py,geoserver/security.py). - Only then: stop defaulting
get_dataset_workspace()toDEFAULT_WORKSPACEfor remote datasets, and audit its call sites (thumbs/thumbnails.py,geoserver/manager.py,geoserver/security.py). - Set
workspaceto NULL for remote datasets; and change the ArcGISalternatederivation, which currently buildsf"{workspace}:{name}"(harvesting/harvesters/arcgis.py:209,311).
Note: this is a pre-existing bug, not introduced by #14381 - reproducible with a single remote GeoNode service and no duplicate URLs.
TBD if needs backport
Two notes on process: worth labelling this security-adjacent given the rule-deletion path, and step 1 is a behavior change for every existing GeoNode-harvested dataset (they'd flip to sourcetype=REMOTE), so it may want a data migration.
Activity
[Fixes #14524] Remote-resource detection is inconsistent across harve…
[Fixes #14524] Remote-resource detection is inconsistent across harve…
[Fixes #14524] Remote-resource detection is inconsistent across harve…