Add Meala as an additional upload backend

CMakeLists.txt

@@ -34,6 +34,7 @@ add_executable(${PROJECT_NAME}
     src/Log.cpp
     src/LogDatabase.cpp
     src/LogLoader.cpp
+    src/MealaUploadTarget.cpp
     src/Sqlite.cpp
     src/StatusBoard.cpp
     src/UploadTarget.cpp)

README.md

@@ -1,6 +1,6 @@
 ![image](logloader_logo.png)
 
-Downloads flight logs from the vehicle over MAVLink FTP and uploads them to a local Flight Review and, optionally, a remote one. Works with PX4 (`.ulg`) and ArduPilot (`.BIN`).
+Downloads flight logs from the vehicle over MAVLink FTP and uploads them to a local Flight Review and, optionally, a remote server. Works with PX4 (`.ulg`) and ArduPilot (`.BIN`).
 
 ### What it fetches, and when
 
@@ -59,6 +59,36 @@ Downloads are staged in a temporary directory and only moved next to the finishe
 
 Older versions kept a database per server and identified logs by the timestamp `LOG_ENTRY` reported, which MAVLink FTP cannot reproduce. On first start `local_server.db` and `remote_server.db` are imported into a single `logloader.db` and their rows matched against the FTP listing by size, so logs already downloaded and uploaded are not fetched or uploaded a second time. The old files are left untouched.
 
+### Upload backends
+
+A target's `backend` says which upload API it speaks. It is per target, so the local Flight Review on the companion and the remote server do not have to be the same kind of thing.
+
+| `backend` | Server | Auth | Notes |
+| --- | --- | --- | --- |
+| `flight_review` (default) | [Flight Review](https://github.com/PX4/flight_review), e.g. [review.px4.io](https://review.px4.io) | None, or `api_key` on an authenticated instance | Records the `/plot_app?log=<uuid>` path it redirects to, which is what lets the UI link to the plot. `.BIN` is not accepted by review.px4.io |
+| `meala` | [Meala](https://apisdynamics.ca) ([API docs](https://apisdynamics.ca/apidocs/)) | `credentials_file`, required | Logs in for a session cookie, then uploads in 5 MB Dropzone-style chunks. Returns no per-log url, so nothing is recorded to link to |
+
+Switching a target's backend does not re-upload what is already up: a target is identified in the database by its name (`local`, `remote`), not by where it points.
+
+To upload to Meala, sign up, download the account's JSON credentials, and point a target at it:
+
+```toml
+[upload_remote]
+enabled = true
+backend = "meala"
+url = "https://apisdynamics.ca"
+credentials_file = "/home/pilot/.config/ark/logloader/meala_creds.json"
+
+# Filled in on every log this target uploads.
+comment = "Uploaded by logloader"
+battery = ""
+pic = ""
+gso = ""
+vehicle_id = ""
+```
+
+The credentials file is the one Meala hands out, `{"username": "...", "token": "..."}`, where the token is the API token from the account page. Give an absolute path: like every other path in the config, it is taken as written and `~` is not expanded. Meala has no anonymous upload, so a `meala` target with no `credentials_file` is disabled at startup with a warning rather than failing one login per upload pass. `email`, `public` and `api_key` are Flight Review's and are ignored.
+
 ### Configuration
 
 `config.toml`, in full, is documented inline in the shipped file. The keys:
@@ -78,6 +108,9 @@ Older versions kept a database per server and identified logs by the timestamp `
 | `upload.interval` | `10` | Seconds between upload passes |
 | `upload_local.*` | enabled, `http://127.0.0.1:5006` | Flight Review on the companion |
 | `upload_remote.*` | disabled, `https://review.px4.io` | `url`, `email`, `public`, `api_key` |
+| `*.backend` | `flight_review` | Upload API this target speaks; see above |
+| `*.credentials_file` | `""` | Meala account credentials; required by that backend |
+| `*.comment` / `battery` / `pic` / `gso` / `vehicle_id` | `Uploaded by logloader` / `""` | Meala form fields sent with every upload |
 
 Tables are one level deep on purpose: ARK-OS's config editor renders exactly that, and a setting an operator cannot reach from the web UI may as well not exist.
 
@@ -152,4 +185,4 @@ make install
 
 ### Future developments
 - Resume interrupted transfers rather than restarting them.
-- Multiple backends: e.g. RobotoAI, DroneLogbook, Auterion Suite, Aloft.
+- Further upload backends: e.g. RobotoAI, DroneLogbook, Auterion Suite, Aloft.

config.toml

@@ -61,3 +61,32 @@ public = false
 # Per-account key for an authenticated Flight Review. When empty no
 # authorization headers are sent at all, which is what open servers expect.
 api_key = ""
+
+# Which upload API this target speaks: "flight_review" (the default) or
+# "meala". Only the transport changes -- the target is still "remote"
+# everywhere else, so switching backends does not re-upload what is already up.
+#
+# For Meala, point url at the Meala server and credentials_file at the JSON
+# credentials downloaded from your Meala account:
+#
+#     backend = "meala"
+#     url = "https://apisdynamics.ca"
+#     credentials_file = "/home/pilot/.config/ark/logloader/meala_creds.json"
+#
+# Paths are taken as written, so give an absolute one -- "~" is not expanded.
+#
+# Meala has no anonymous upload, so a meala target without credentials_file is
+# disabled at startup. email, public and api_key are Flight Review's and are
+# ignored by it.
+backend = "flight_review"
+credentials_file = ""
+
+# Meala only: sent with every log this target uploads, as the flight's details
+# on Meala. Edit these to match the aircraft and crew; empty fields are sent
+# empty. comment is for observations or descriptions, pic is the Pilot in
+# Command, gso the Ground Station Operator or other second crew
+comment = "Uploaded by logloader"
+battery = ""
+pic = ""
+gso = ""
+vehicle_id = ""

src/Config.cpp

@@ -1,6 +1,7 @@
 #include "Config.hpp"
 
 #include <algorithm>
+#include <cctype>
 #include <cstdlib>
 #include <filesystem>
 #include <sstream>
@@ -63,6 +64,96 @@ T value_or(const Preferred& preferred, const Legacy& legacy, T fallback)
 
 } // namespace
 
+bool parse_upload_backend(const std::string& text, UploadBackend& backend)
+{
+	std::string key;
+
+	// Accept "flight_review", "flight-review" and "flightreview" alike; the
+	// separator is the kind of thing an operator gets wrong once per install.
+	for (const char c : text) {
+		if (c != '_' && c != '-' && c != ' ') {
+			key += static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
+		}
+	}
+
+	if (key == "flightreview") {
+		backend = UploadBackend::FlightReview;
+		return true;
+	}
+
+	if (key == "meala") {
+		backend = UploadBackend::Meala;
+		return true;
+	}
+
+	return false;
+}
+
+const char* to_string(UploadBackend backend)
+{
+	switch (backend) {
+	case UploadBackend::Meala:
+		return "meala";
+
+	case UploadBackend::FlightReview:
+	default:
+		return "flight_review";
+	}
+}
+
+namespace
+{
+
+// Reads one target's backend, warning rather than throwing on a typo: an
+// unreadable backend name should not stop the daemon from starting with the
+// default one.
+void load_backend(const toml::node_view<toml::node>& table, UploadTargetConfig& target)
+{
+	const auto text = table["backend"].value<std::string>();
+
+	if (!text.has_value() || text->empty()) {
+		return;
+	}
+
+	if (!parse_upload_backend(*text, target.backend)) {
+		LOG_WARN("Unknown backend \"" << *text << "\" for upload target " << target.name
+			 << ", using " << to_string(target.backend));
+	}
+}
+
+// Meala's form fields are free text, but an operator writing vehicle_id = 3
+// means the same as vehicle_id = "3"; refusing the bare number would silently
+// send an empty field instead.
+std::string text_or(const toml::node_view<toml::node>& node, const std::string& fallback)
+{
+	if (auto value = node.value<std::string>(); value.has_value()) {
+		return trim(*value);
+	}
+
+	if (auto value = node.value<int64_t>(); value.has_value()) {
+		return std::to_string(*value);
+	}
+
+	if (auto value = node.value<double>(); value.has_value()) {
+		std::ostringstream text;
+		text << *value;
+		return text.str();
+	}
+
+	return fallback;
+}
+
+void load_meala_fields(const toml::node_view<toml::node>& table, UploadTargetConfig& target)
+{
+	target.meala_comment = text_or(table["comment"], target.meala_comment);
+	target.meala_battery = text_or(table["battery"], target.meala_battery);
+	target.meala_pic = text_or(table["pic"], target.meala_pic);
+	target.meala_gso = text_or(table["gso"], target.meala_gso);
+	target.meala_vehicle_id = text_or(table["vehicle_id"], target.meala_vehicle_id);
+}
+
+} // namespace
+
 std::string resolve_config_path(int argc, char** argv)
 {
 	for (int i = 1; i < argc; i++) {
@@ -147,13 +238,19 @@ Config load_config(const std::string& path)
 	config.local.public_logs = local["public"].value_or(true);
 	config.local.email = local["email"].value_or("");
 	config.local.api_key = trim(local["api_key"].value_or<std::string>(""));
+	load_backend(local, config.local);
+	config.local.credentials_file = trim(local["credentials_file"].value_or<std::string>(""));
+	load_meala_fields(local, config.local);
 
 	config.remote.name = kTargetRemote;
 	config.remote.url = value_or<std::string>(remote["url"], file["remote_server"], "https://review.px4.io");
 	config.remote.enabled = value_or<bool>(remote["enabled"], file["upload_enabled"], false);
 	config.remote.public_logs = value_or<bool>(remote["public"], file["public_logs"], false);
 	config.remote.email = value_or<std::string>(remote["email"], file["email"], "");
 	config.remote.api_key = trim(value_or<std::string>(remote["api_key"], file["remote_api_key"], ""));
+	load_backend(remote, config.remote);
+	config.remote.credentials_file = trim(remote["credentials_file"].value_or<std::string>(""));
+	load_meala_fields(remote, config.remote);
 
 	if (config.local.url.empty()) {
 		config.local.enabled = false;
@@ -163,6 +260,17 @@ Config load_config(const std::string& path)
 		config.remote.enabled = false;
 	}
 
+	// Meala authenticates every upload, so a target without credentials would
+	// fail the login once per pass forever. Saying so at startup beats one
+	// warning per upload interval.
+	for (UploadTargetConfig* target : {&config.local, &config.remote}) {
+		if (target->enabled && target->backend == UploadBackend::Meala && target->credentials_file.empty()) {
+			LOG_WARN("Upload target " << target->name << " uses the meala backend but has no "
+				 "credentials_file; disabling it");
+			target->enabled = false;
+		}
+	}
+
 	if (g_used_legacy_key) {
 		LOG_WARN(path << " uses the pre-overhaul flat key layout. It still works, but the "
 			 "shipped config.toml shows the current one and the fallback will be removed.");

src/Config.hpp

@@ -10,15 +10,41 @@
 constexpr const char* kTargetLocal = "local";
 constexpr const char* kTargetRemote = "remote";
 
+// Which upload API a target speaks. The name is a target's identity in the
+// database, so it stays "remote" whatever it is pointed at; this is how the
+// bytes get there.
+enum class UploadBackend {
+	FlightReview,
+	Meala,
+};
+
+// Parses the config spelling ("flight_review", "meala"). False leaves the
+// backend untouched and tells the caller the text was not recognised.
+bool parse_upload_backend(const std::string& text, UploadBackend& backend);
+
+const char* to_string(UploadBackend backend);
+
 struct UploadTargetConfig {
 	std::string name;
 	bool enabled {false};
+	UploadBackend backend {UploadBackend::FlightReview};
 	std::string url;
 	std::string email;
 	// Per-account key for authenticated Flight Review instances. Empty means
 	// no auth headers are sent at all, which is what open servers expect.
 	std::string api_key;
+	// Meala only: JSON file holding the account's username and token, as
+	// downloaded from the Meala account page. Meala has no anonymous upload, so
+	// a target without this cannot be enabled.
+	std::string credentials_file;
 	bool public_logs {false};
+	// Meala only: what each upload fills its form fields with. Meala requires
+	// every field to be present, so empty is sent as an empty field.
+	std::string meala_comment {"Uploaded by logloader"};
+	std::string meala_battery;
+	std::string meala_pic;
+	std::string meala_gso;
+	std::string meala_vehicle_id;
 };
 
 struct Config {

src/LogLoader.cpp

@@ -53,7 +53,7 @@ LogLoader::LogLoader(const Config& config)
 	fs::create_directories(_config.logs_directory, ec);
 
 	for (const auto* target : _config.targets()) {
-		_targets.push_back(std::make_unique<UploadTarget>(*target));
+		_targets.push_back(make_upload_target(*target));
 	}
 }