Fleet Management Integration, Teltonika Telematics Device Support #1147

pankalog · 2023-10-27T14:46:06Z

This is still missing tests, which I plan to implement this week.

The UI folder will be removed from the PR before merging, it is just useful for debugging purposes.

and either generates or updates an asset depending on the IMEI of the device.

and either generates or updates an asset depending on the IMEI of the device. Currently reads Teltonika payloads, finds the Asset using the IMEI, and determines whether or not a new Asset needs to be created or an old one needs to be updated.

Creates asset when new IMEI is used, Updates the asset if the IMEI has already been used.

The method will parse the payload coming from the device to decode it into an attribute list.

This asset helps with conciseness on code in the MQTT Handler, since we are aware that the required attributes exist and are complete.

Parses the JSON payload according to the screenshot on Teltonika forums, links the parameter IDs to their descriptions based on the Teltonika wiki, and dynamically inserts or updates an asset with the most recent data. TODO: - Proper Type definition for each attribute depending on the `Type` parameter in the Teltonika parameter ID forums - Secure communication and provisioning of Teltonika Devices - Automatically retrieve Teltonika device parameter information (preferrably from an official API)

When an asset marker is clicked, its history from midnight to the current time is displayed as lines and points. When deselected, the GeoJSON layers and sources are removed from the MapLibre map object. TODO: Improve overall type-safety and robustness, improve UI line and point elements, implement configuration item to specifically allow the overlay to be visible.

Overlay is properly removed when another asset marker is selected, and when no asset is selected.

- Now does not require authentication for publishing to Teltonika topic (while Authentication is being worked on) - Improved robustness and added more logging for when devices connect and publish.

- Since I have a device in my hands, I can test out the payload, and adjust the jackson parser.

- FMC003.json is not being included in the build, quick and dirty fix to get the parameters to be read accordingly.

- include FMC003.json in the build, quick and dirty fix to get the parameters to be read accordingly.

- Directory fixes

- Directory fixes and change string

- Fixed bug that only read the first attribute; - Now logs the amount of Teltonika AVL parameters from file

- Fixed latitude and longitude regex to properly recognize all coordinates

- Fixed latitude and longitude regex to properly recognize all coordinates - Select the Europe MBTILES file for deployment

- Correctly parse the coordinates sent by the device

- MQTT Handler updated to correctly parse data from real devices - State.java now correctly parses the Type of each parameter - State.java now correctly includes Attribute Metadata, based on TeltonikaParameter - MQTT Handler now correctly handles updating and creating an Asset's Attributes using SEDA messages

- Bug when trying to render a Unit of Measurement that is not defined in the Constants file

- Support for sending messages to the Teltonika devices. - Bug fixes and refactors

- Formatted the command to be of the appropriate format for the device

- Bug fixes

…y of Teltonika datapoints

* New Nominatim client for better support * Fixed formatting for Overview Analysis Type

Analysis types now more statically defined and easier to deal with. Functionality for multiple analysis types to come.

Reworked to include the ref's attribute's type, used for determining what types of analysis should be available, and for casting/display purposes.

Remove the connectedCallback() actions, as they were being triggered two times, once for the connectedCallback and once for firstUpdated. firstUpdated suffices, we just initialize everything later down the lifecycle of the LitElement

Remove a lot of console.logs and also updated UI element to explain the reasoning for the displayed error.

This fix now properly updates the attributes within AttributeActionEvent:detail, so that the attribute included is the actual one that is selected

EVEN IF you handle the exception, it still prints it, taking up ENORMOUS amounts of space in my log. Just destroys the great work I've done over months of printing ONE LINE per correctly parsed payload. Very happy to have taken the executive decision for this. Took some emotional strength, but I finally am committing the removal of this messy-for-my-logs piece of code.

…ement

Theoretically this should fix everything?

richturner · 2024-05-14T18:45:14Z

manager/src/main/resources/logging-dev.properties

+org.openremote.manager.mqtt.level=FINE

 # Processing time logging
-#org.openremote.manager.asset.AssetProcessingService.level=FINE
-#org.openremote.manager.asset.AssetStorageService.level=FINE
+org.openremote.manager.asset.AssetProcessingService.level=WARNING
+org.openremote.manager.asset.AssetStorageService.level=WARNING


Not sure why this is being changed in this PR...logging-dev is for development use and these levels were set to aid in tracing issues

richturner · 2024-05-14T18:45:45Z

model/src/main/java/org/openremote/model/util/ValueUtil.java

@@ -216,7 +216,7 @@ public static <T> Optional<T> parse(String jsonString, Type type) {
 try {
 return Optional.of(JSON.readValue(jsonString, JSON.constructType(type)));
 } catch (Exception e) {
- LOG.log(Level.WARNING, "Failed to parse JSON", e);
+// LOG.log(Level.WARNING, "Failed to parse JSON", e);


Wouldn't merge this

richturner · 2024-05-14T18:48:11Z

ui/app/manager/src/pages/page-map.ts

+ this._assets.filter((asset) => {
+ if (!asset.attributes) {
+ return false;
+ }
+ const attr = asset.attributes[WellknownAttributes.LOCATION] as Attribute<GeoJSONPoint>;
+ return !attr.meta || !attr.meta.hasOwnProperty(WellknownMetaItems.SHOWONDASHBOARD) || !!Util.getMetaValue(WellknownMetaItems.SHOWONDASHBOARD, attr);
+ })
+ .sort((a, b) => {
+ if (a.attributes[WellknownAttributes.LOCATION].value && b.attributes[WellknownAttributes.LOCATION].value) {
+ return b.attributes[WellknownAttributes.LOCATION].value.coordinates[1] - a.attributes[WellknownAttributes.LOCATION].value.coordinates[1];
+ } else {
+ return;
+ }
+ })
+ .map(asset => {
+ return html`
+ <or-map-marker-asset
+ ?active="${this._currentAsset && this._currentAsset.id === asset.id}"
+ .asset="${asset}"
+ .config="${this.config.markers}"></or-map-marker-asset>
+ `;
+ })


Reformatting here makes it difficult to follow what has actually changed...we do need to enforce some linting and coding standards though

richturner · 2024-05-14T18:56:23Z

ui/app/manager/src/pages/page-map.ts

+ <or-map-location-history-overlay .assetId="${this._currentAsset?.id}" .map="${this._map}" ></or-map-location-history-overlay>
+${currentAssetSelected ? html`
+ <or-map-asset-card .config="${this.config?.card}" .assetId="${this._currentAsset.id}"
+ .markerconfig="${this.config?.markers}" .map="${this._map}"></or-map-asset-card>


This newly added map property doesn't appear to be used

richturner · 2024-05-14T19:00:31Z

ui/app/manager/src/pages/page-map.ts

+<!-- 
+ <or-map-location-history-markers .assetId="${this._currentAsset?.id}" .map="${this._map}"></or-map-location-history-markers>
+ -->
+ <or-map-location-history-overlay .assetId="${this._currentAsset?.id}" .map="${this._map}" ></or-map-location-history-overlay>


If this location history is to be integrated into the generic manager app then it will need to be configurable

richturner · 2024-05-14T19:01:06Z

ui/component/or-attribute-card/src/index.ts

+ type: "all",
 fromTimestamp: this._startOfPeriod,
 toTimestamp: this._endOfPeriod,
 amountOfPoints: 20,
- } as AssetDatapointLTTBQuery
+ } as AssetDatapointAllQuery


Is this intentional?

To additionally comment on this; we're intentionally using LTTB in the or-attribute-card here.

Thanks but is it a change that makes sense for everything or just this fleet management application? If it doesn't make sense in general then it needs to be configurable on the map config

richturner · 2024-05-15T08:45:12Z

ui/component/or-dashboard-builder/src/panels/attributes-panel.ts

@@ -193,7 +193,7 @@ export class AttributesPanel extends LitElement {
 }

 protected async loadAssets(): Promise<Asset[]> {
- if(this.attributeRefs.filter(ar => !this.getLoadedAsset(ar)).length > 0) {
+ if(true) {


Remove or fix

richturner · 2024-05-15T09:31:36Z

ui/component/or-dashboard-builder/src/util/nominatim-browser.ts

I think we can just update the forward/reverse geocoder functions in mapwidget rather than this, as it doesn't add enough functionality to warrant the copy paste and source code isn't maintained. We should definitely default to nominatim openstreetmap API though.

richturner · 2024-05-15T09:33:42Z

ui/component/or-dashboard-builder/src/widgets/session-widget.ts

Need a way of being able to plugin custom widgets at runtime from a custom project

richturner · 2024-05-15T09:36:27Z

ui/component/or-map/src/mapwidget.ts

+ public async getGeoJsonFeature(query: {query: string}){
+ // let gc: MaplibreGeocoder. = this._geocoder;
+ }


Does this not work? Can it be fixed or removed or at least commented out with reason

richturner · 2024-05-15T09:37:22Z

ui/component/or-map/src/mapwidget.ts

+ public async getAddress(query: {lat: number, lon: number}){
+ // let gc: MaplibreGeocoder. = this._geocoder;
+ return this._reverseGeocode(query);
+ }
+
+


Maybe should be exposed from the or-map so no dependency on the map for geocoding

richturner · 2024-05-15T09:39:43Z

ui/component/or-map/src/markers/or-map-asset-card-trip-section.ts

This needs to be a pluggable mechanism from a custom project

richturner · 2024-05-15T09:44:32Z

ui/component/or-map/src/or-map-location-history-markers.ts

+ const popup = new maplibregl.Popup().setText(
+ 'Construction on the Washington Monument began in 1848.'


Cleanup required

richturner · 2024-05-15T09:47:32Z

ui/component/or-map/src/or-map-location-history-markers.ts

I like this as a marker type but needs to be configurable at the map level.

Can this marker not extend the OrMapMarkerAsset or even just add this functionality to that marker.

History duration to retrieve in map config (if set then uses this marker instead of asset marker)

richturner · 2024-05-15T09:51:14Z

ui/component/or-map/src/orMapLocationHistoryOverlay.ts

Not clear to me why this is needed rather than encapsulating everything in the history marker

pankalog and others added 30 commits September 8, 2023 15:12

Reads Teltonika JSON payloads

0dab012

and either generates or updates an asset depending on the IMEI of the device.

Reads Teltonika JSON payloads

d98e63d

and either generates or updates an asset depending on the IMEI of the device. Currently reads Teltonika payloads, finds the Asset using the IMEI, and determines whether or not a new Asset needs to be created or an old one needs to be updated.

Unsecured auto-provisioning.

6c7847b

Creates asset when new IMEI is used, Updates the asset if the IMEI has already been used.

New method for parsing Attributes from payload

7cf398f

The method will parse the payload coming from the device to decode it into an attribute list.

New method for parsing Attributes from payload

a820098

The method will parse the payload coming from the device to decode it into an attribute list.

First implementation of the Car Asset

262a0f2

This asset helps with conciseness on code in the MQTT Handler, since we are aware that the required attributes exist and are complete.

Location History Overlay V1.01

bb0d359

Overlay is properly removed when another asset marker is selected, and when no asset is selected.

Tester commit

6dc425b

Revert map chart changes

01af21d

MQTT Handler Improvements

dd27e90

- Now does not require authentication for publishing to Teltonika topic (while Authentication is being worked on) - Improved robustness and added more logging for when devices connect and publish.

Teltonika MQTT Handler Improvements

87ee6ab

- Since I have a device in my hands, I can test out the payload, and adjust the jackson parser.

Update .env

b45576a

Update docker-compose.yml

868b7a5

Teltonika MQTT Handler Improvements

4ea5910

- FMC003.json is not being included in the build, quick and dirty fix to get the parameters to be read accordingly.

Teltonika MQTT Handler Improvements

7671f62

- include FMC003.json in the build, quick and dirty fix to get the parameters to be read accordingly.

Teltonika MQTT Handler Improvements

503715d

- Directory fixes

Teltonika MQTT Handler Improvements

271bbd1

- Directory fixes and change string

Teltonika MQTT Handler Improvements

60b9213

- Fixed bug that only read the first attribute; - Now logs the amount of Teltonika AVL parameters from file

Teltonika MQTT Handler Improvements

06e92fa

- Fixed latitude and longitude regex to properly recognize all coordinates

Teltonika MQTT Handler Improvements

d3cdb91

- Fixed latitude and longitude regex to properly recognize all coordinates - Select the Europe MBTILES file for deployment

Teltonika MQTT Handler Improvements

2c7b1cf

- Correctly parse the coordinates sent by the device

Parameter Type Parser WIP

ee7189c

Payload State Parser Bug Fixes

c519cc3

- Bug when trying to render a Unit of Measurement that is not defined in the Constants file

Updated docker-compose.yml to use mbtiles file for Europe

ae6117b

TeltonikaMQTTHandler V2.5

88f8851

- Support for sending messages to the Teltonika devices. - Bug fixes and refactors

TeltonikaMQTTHandler V2.5

8e2f0cb

- Formatted the command to be of the appropriate format for the device

TeltonikaMQTTHandler V2.5

ef27fa9

- Bug fixes

pankalog added 15 commits January 15, 2024 22:49

Use AssetDatapointAllQuery instead of LTTB for more accurate displa…

db60adb

…y of Teltonika datapoints

Better Reverse Geocode support and formatting

6b56b94

* New Nominatim client for better support * Fixed formatting for Overview Analysis Type

Fix Analysis Types

465f13f

Analysis types now more statically defined and easier to deal with. Functionality for multiple analysis types to come.

Update import

e520907

Rework Input/Output refs

cbdd893

Reworked to include the ref's attribute's type, used for determining what types of analysis should be available, and for casting/display purposes.

Stop double-rendering

99b00cd

Remove the connectedCallback() actions, as they were being triggered two times, once for the connectedCallback and once for firstUpdated. firstUpdated suffices, we just initialize everything later down the lifecycle of the LitElement

Improve UI and console logs

0a819e8

Remove a lot of console.logs and also updated UI element to explain the reasoning for the displayed error.

Attributes Panel Fix

0b1b4f0

This fix now properly updates the attributes within AttributeActionEvent:detail, so that the attribute included is the actual one that is selected

Fix console logs

474b54a

Merge branch 'master' into feature/fleet-management

522204f

Merge branch 'master' into feature/fleet-management

d6efabd

Merge remote-tracking branch 'origin/master' into feature/fleet-manag…

f3865fd

…ement

Merge branch 'master' into feature/fleet-management

9423d58

Update to latest OpenRemote submodule

d241b8c

Theoretically this should fix everything?

richturner reviewed May 14, 2024

View reviewed changes

richturner reviewed May 15, 2024

View reviewed changes

ui/component/or-map/src/markers/or-map-asset-card-trip-section.ts

Copy link

Member

richturner May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be a pluggable mechanism from a custom project

richturner reviewed May 15, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fleet Management Integration, Teltonika Telematics Device Support #1147

Fleet Management Integration, Teltonika Telematics Device Support #1147

pankalog commented Oct 27, 2023 •

edited

richturner May 14, 2024

richturner May 14, 2024

richturner May 14, 2024

richturner May 14, 2024

richturner May 14, 2024

richturner May 14, 2024

MartinaeyNL May 15, 2024

richturner May 15, 2024 •

edited

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

richturner May 15, 2024

		const popup = new maplibregl.Popup().setText(
		'Construction on the Washington Monument began in 1848.'

Fleet Management Integration, Teltonika Telematics Device Support #1147

Are you sure you want to change the base?

Fleet Management Integration, Teltonika Telematics Device Support #1147

Conversation

pankalog commented Oct 27, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richturner May 15, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pankalog commented Oct 27, 2023 •

edited

richturner May 15, 2024 •

edited