# SyncTide v1.9.0 — Update Log **Release date:** 2026-04-21 **Schema version:** 21 (migrations `020_protocol_ingestion.sql` + `021_extend_data_types.sql` required) **Previous version:** v1.8.2 SyncTide now reads industrial devices directly over the network. Three protocols are live and tested end-to-end: Modbus TCP, OPC UA, and IEC 60870-5-104. CSV ingestion stays the default for customers who already drop files in a folder. --- ## What's new ### Multi-protocol ingestion - **Modbus TCP** (live). Per-device config: `host`, `port`, `unit_id`, `timeout_ms`, `address_offset`, `byte_order`, optional `batch_reads` / `batch_gap_max`. Data types: `int16`, `uint16`, `int32`, `uint32`, `float`, `bool`, **`int64`**, **`uint64`**, **`double`** (4 consecutive registers for 64-bit). Addresses accept classic Modicon syntax (`40001`) or explicit prefixes (`hr:100`, `ir:0`, `di:5`, `coil:12`). - **OPC UA** (live). Per-device config: `endpoint_url`, optional `username` / `password`. NodeId addresses: `ns=2;s=TagName` or `ns=2;i=1234`. Int64 / UInt64 / Double supported natively by the UA type system. - **IEC 60870-5-104** (live). Per-device config: `host`, `port` (default 2404), `common_address`, `originator_address`, `time_tagged` (default true). **Stateful driver with event-driven ingestion**: every spontaneous ASDU the RTU emits lands in `measurements` with its CP56Time2a timestamp, including buffered events replayed after a reconnect. 64-bit types rejected at the driver level — no 64-bit ASDU variant exists in the standard. - **DNP3** — **deferred**. No viable maintained Python library currently exists (`pydnp3` is abandoned, cp27/35/36 only). Will be revisited when one ships, or if we wrap `opendnp3` (C++) via cffi. - **CSV** (legacy, still the default) — now supports a per-device `connection_config.folder_path` override. Leave blank to keep the historical `raw_data//` behaviour. ### Timestamp fidelity (device-side, not poll-side) - **OPC UA**: samples are stored with `SourceTimestamp` from the server's DataValue — the time the server last sampled the underlying source, not the master poll time. Unchanged values deduplicate automatically (ON CONFLICT on `(device_id, time_stamp, tag_name)`), so static tags don't blow up the measurements table. - **IEC 60870-5-104**: samples carry the CP56Time2a timestamp the RTU stamped on the ASDU. Buffered-event replay after reconnect imports with the **original** measurement time, not the reconnect moment. - **Modbus TCP**: the protocol carries no timestamp, so every reading is stored with the master poll time. The UI explicitly calls this out on the Communication tab so customers understand the trade-off. ### Batched reads (new in v1.9.0) - **Modbus TCP driver**: tags are automatically coalesced into a single request per contiguous address span, respecting the protocol's 125-register / 2000-bit per-PDU caps. A 500-tag packed register map now polls in **4 requests instead of 500** — a measured ~125× speedup on the synthetic load test. - **OPC UA driver**: all tags on a device are fetched in a **single** UA Read service call (`client.read_attributes()`). Per-node failures log a warning and skip that tag instead of aborting the whole cycle. - No user config required. Defaults assume batched reads are on with a 16-register gap tolerance. `batch_reads = false` is an escape hatch for the rare Modbus device that rejects multi-register reads. ### Modbus byte-order (new in v1.9.0) - Per-device `byte_order` setting picks how the device lays out 32- and 64-bit values: **ABCD** (classic big-endian, Schneider/Modicon default), **CDAB** (word-swap, Honeywell/Eaton), **BADC** (byte-swap within words), **DCBA** (full little-endian, some Allen-Bradley and Siemens gear). Defaults to ABCD for backwards compatibility; single- register and bool tags are unaffected. ### One worker per protocol - `modbus_watchdog.py` and `opcua_watchdog.py` each run as a dedicated process, managed by `platform_supervisor.py`. A misbehaving driver can no longer stall the others or block CSV ingestion. - Each worker writes to its own rotating log in `logs/_poller.log`. - Per-device poll interval, **1–86400 seconds**, configurable independently on every non-CSV device. ### Licensing - New module flag `protocol_ingestion` — required for any non-CSV polling. Existing CSV-only customers are unaffected and need no licence change. ### Python runtime → 3.13.13 - Several SCADA protocol libraries (`asyncua`, `c104`) do not yet publish `cp314` wheels, so the runtime is intentionally pinned. - Embedded Python in the installer is now `python-3.13.13-embed-amd64.zip`. - `installer/scripts/post_install.pas` patches `python313._pth` (was `python314._pth`). ### Equipment Configuration page - New **"Add device"** expander. - New **"Communication" tab**: protocol selector, endpoint config form, per-tag address editor, and a **"Test connection"** button that probes the device without persisting anything. - **Tag editor** gets per-row delete buttons, CSV export (download current tag map), a blank CSV template download, and CSV import with replace-all / merge-by-tag-name modes for bulk-editing large register maps in Excel. - **Timestamp hint banner** on the Communication tab: OPC UA and IEC 104 devices show the RTU-side-timestamp note (including buffered- replay behaviour); Modbus devices show the poll-time explanation. ### New backend endpoints - `POST /admin/devices` - `PUT /admin/devices/{id}/comm-config` - `GET /admin/device-tag-addresses/{id}` - `PUT /admin/device-tag-addresses/{id}` - `POST /admin/devices/{id}/test-connection` ### Migrations - **020 — protocol ingestion**. `devices` gains `protocol_type`, `connection_config` (JSONB), `poll_interval_seconds`, `polling_enabled`, `last_poll_at`, `last_poll_status`, `last_poll_error`. Defaults keep existing rows on `protocol_type='csv'`. New table `device_tag_addresses` (per-tag register / NodeId map with `data_type`, `scale`, `offset_val`, `unit`, `enabled`). Check constraints: `protocol_type IN ('csv','modbus_tcp','opcua','iec104','dnp3')` and `poll_interval_seconds BETWEEN 1 AND 86400`. - **021 — 64-bit tag types**. Widens the `data_type` CHECK constraint on `device_tag_addresses` to accept `int64`, `uint64`, `double`. --- ## Pending (not blocked by this release) - DNP3 driver (phase 5 — deferred until a viable Python library exists or we commit to a cffi wrapper of opendnp3). - TLS / certificate auth for OPC UA (roadmap). - Auto-discovery of tags on supported devices (roadmap). - OPC UA HistoryRead support for full historical replay (not just live reads with SourceTimestamp). --- ## Files changed | File | Change | Recompile | |------|--------|-----------| | `migrations/020_protocol_ingestion.sql` | New — protocol columns + `device_tag_addresses` | No | | `migrations/021_extend_data_types.sql` | New — widens data_type CHECK to include int64/uint64/double | No | | `protocol_drivers/base.py` | New — driver interface | Yes | | `protocol_drivers/common.py`, `poller.py` | New — shared poller loop + helpers | Yes | | `protocol_drivers/modbus_tcp.py` | New — Modbus TCP driver with batching + byte-order + 64-bit | Yes | | `protocol_drivers/opcua.py` | New — OPC UA driver with batched `read_attributes()` | Yes | | `protocol_drivers/iec104.py` | New — IEC 60870-5-104 stateful event-driven driver | Yes | | `modbus_watchdog.py`, `opcua_watchdog.py`, `iec104_watchdog.py` | New — one worker process per protocol | Yes | | `platform_supervisor.py` | Spawns + restarts protocol pollers via `self.protocol_pollers` dict | Yes | | `main.py` | New `/admin/devices`, `/admin/device-tag-addresses`, `/admin/devices/{id}/test-connection`, `protocol_ingestion` gate, 64-bit data type validation | Yes | | `api_client.py` | Client helpers for the new endpoints | Yes | | `license_manager.py` | `is_protocol_ingestion_allowed()` + `protocol_ingestion` module flag | Yes | | `tools/generate_license.py` | `protocol_ingestion` module added to the generator's ALL_MODULES | No | | `tools/modbus_test_server.py`, `opcua_test_server.py`, `iec104_test_server.py` | New — local RTU simulators for smoke-testing | No | | `ingest_csvs.py` | Per-device `connection_config.folder_path` override for CSV devices | Yes | | `pages/7_Equipment_Configuration.py` | Add-device expander, Communication tab, tag-address editor with CSV import/export + per-row delete, timestamp hint banner | No | | `i18n.py` | ~70 new keys (en + pt-PT) across all new UI | Yes | | `installer/scripts/post_install.pas` | `python313._pth` (was `python314._pth`) | No (Pascal) | | `installer/synctide.iss` | Embedded Python 3.13.13 asset | No (Inno) | | `installer/build_installer.py` | `PYTHON_VERSION = "3.13.13"` | No | | `install_local.ps1` | `Test-Python313Plus`, Python 3.13.13 candidates | No | | `setup_build.py` | Adds all protocol driver modules + watchdogs to the Cython build | No | | `requirements.txt` | `pymodbus==3.6.9`, `asyncua==1.1.8`, `c104==2.2.1` | No | | `version.py` | `__version__ = 1.9.0`, `SCHEMA_VERSION = 21`, `BUILD_DATE = 2026-04-21` | Yes | --- ## Deployment Installer: run `SyncTideSetup-1.9.0.exe` on an existing install — the auto-upgrade path stops services, swaps files, applies migration 020 on next startup, and preserves DB / admin / licence. Manual: stop services → copy updated `.pyd` files + new `protocol_drivers/` folder → update `integrity_manifest.json` with new SHA-256 hashes → run `_setup/run_migrations.py` (applies 020 and 021) → start services → verify `/version` returns `1.9.0` with `SCHEMA_VERSION: 21` and `/license/status` lists the `protocol_ingestion` module (if your licence includes it). CSV-only customers require no licence re-issue. Customers who want to poll Modbus TCP or OPC UA devices need a new `.lic` that includes the `protocol_ingestion` module.