Enable contract listeners with multiple filters #1418
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1418 +/- ##
=======================================
Coverage 99.99% 99.99%
=======================================
Files 321 321
Lines 23027 23079 +52
=======================================
+ Hits 23025 23077 +52
Misses 1 1
Partials 1 1
|
@nguyer (fyi @EnriqueL8) - there is a really good reason why we need this (at least prior to this PR). That's because calculating the signature of a listener is not an easy thing to do - and FireFly doesn't expose any way for code outside of FireFly to do it. So if you think about the common startup reconcile in most applications, where it does a query-then-create on the listeners it needs, you need to be able to match the things you get back, to the things you need. That means you need to be able to create the signature for the things you need (not just the things you create). Currently, we avoid this by having it so that if you try and recreate the new thing with the same signature, it prevents you with a 409. In fact that's important enough that we found and fixed a bug in it recently in #1433. Now I can imaging this PR might provide a proposal for a fundamentally new way to do this, by pushing the complex reconcile logic inside of listener. But we do need a clear migration path, and to know what the impact would be on apps relying on the old behavior - because without any thought they would duplicate a new listener on every startup. |
|
@peterbroadhurst I'm trying to pick this back up but really not sure what direction to take it. I think part of my struggle is that I myself haven't run into the use case that is driving the need for this functionality, so I'm sort of designing a solution to an unknown problem. I could add an updated version of the constraint we used to have (and maybe do some heavy DB schema changes to support that) but what would the constraint be? Topic/Location/[SOMETHING]. Is that last element of the constraint the combined signature of each filter in the listener? Do we need to prevent someone from creating the same exact Topic/Location/[Combination of Signatures] or do we want to prevent the same Topic/Location/[Single event signature] from being used in multiple listeners even if the set of filters in each of those listeners is different. Does that question make sense? I feel like this PR probably needs another round of design discussion before it's ready for more coding. |
Signed-off-by: Nicko Guyer <[email protected]>
Signed-off-by: Nicko Guyer <[email protected]>
Signed-off-by: Nicko Guyer <[email protected]>
This PR adds the ability to listen to multiple types of events on the same contract listener, by adding an array of listeners, rather than a single event signature/location per listener. The old way of creating a listener is still accepted by the API, but it will always be returned in the filters array now. This is a migration concern that needs to be documented.
Open questions
One thing I'm not sure about here, is that this PR as-is removes the uniqueness constraint on listeners by topic/location/signature. It now allows multiples. I'm not sure if this is a problem or not. I can add that constraint back, but it would likely require some more sophisticated DB changes. Which brings me to the next point...
Right now all the filters for a contract listener get serialized to JSON and stored in a single column. I lated realized this means we lose the ability to query/filter (no pun intended) by signature, location, etc. which we used to do, in order to check for duplicates. I'm not sure if this is required or not, but wanted to call it out.
Example
Create contract listener request
{ "filters": [ { "interface": { "id": "aaa0e410-2b5b-4815-a80a-a18f2ae59f7d" }, "eventPath": "BatchPin", "location": { "address": "0xb0cd60ade460e797e0c9d206290ac4ed45672c60" } } ], "name": "CustomBatchPin", "options": { "firstEvent": "oldest" }, "topic": "batch-pin" }Create contract listener response
{ "id": "acc0d227-1da4-4d0d-bbe0-0c60f754158f", "namespace": "default", "name": "CustomBatchPin", "backendId": "018b258a-0c2c-07c0-5d59-50583ae91f1e", "created": "2023-10-12T20:18:06.012167Z", "filters": [ { "event": { "name": "BatchPin", "description": "", "params": [ { "name": "author", "schema": { "type": "string", "details": { "type": "address", "internalType": "address" }, "description": "A hex encoded set of bytes, with an optional '0x' prefix" } }, { "name": "timestamp", "schema": { "oneOf": [ { "type": "string" }, { "type": "integer" } ], "details": { "type": "uint256", "internalType": "uint256" }, "description": "An integer. You are recommended to use a JSON string. A JSON number can be used for values up to the safe maximum." } }, { "name": "action", "schema": { "type": "string", "details": { "type": "string", "internalType": "string" } } }, { "name": "uuids", "schema": { "type": "string", "details": { "type": "bytes32", "internalType": "bytes32" }, "description": "A hex encoded set of bytes, with an optional '0x' prefix" } }, { "name": "batchHash", "schema": { "type": "string", "details": { "type": "bytes32", "internalType": "bytes32" }, "description": "A hex encoded set of bytes, with an optional '0x' prefix" } }, { "name": "payloadRef", "schema": { "type": "string", "details": { "type": "string", "internalType": "string" } } }, { "name": "contexts", "schema": { "type": "array", "details": { "type": "bytes32[]", "internalType": "bytes32[]" }, "items": { "type": "string", "description": "A hex encoded set of bytes, with an optional '0x' prefix" } } } ] }, "location": { "address": "0xb0cd60ade460e797e0c9d206290ac4ed45672c60" }, "interface": { "id": "aaa0e410-2b5b-4815-a80a-a18f2ae59f7d" }, "signature": "BatchPin(address,uint256,string,bytes32,bytes32,string,bytes32[])" } ], "topic": "batch-pin", "options": { "firstEvent": "oldest" } }