Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ldes graphdb connector #65

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Ldes graphdb connector #65

wants to merge 8 commits into from

Commits on Aug 16, 2021

  1. GraphDB connector

    It works with SPARQL queries, so any other triples store will work with it with minor changes
    D34DPlayer committed Aug 16, 2021
    Configuration menu
    Copy the full SHA
    f333837 View commit details
    Browse the repository at this point in the history
  2. Take into account languages and types

    Include @type and @language into the SPARQL query
    Perform a recursive addition if we find an array
    D34DPlayer committed Aug 16, 2021
    Configuration menu
    Copy the full SHA
    101556f View commit details
    Browse the repository at this point in the history
  3. Bring back versioning

    Now done properly, with settings to choose the field to use as "versionOf" and the one used to sort the versions
    D34DPlayer committed Aug 16, 2021
    Configuration menu
    Copy the full SHA
    6b96bd8 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. Connector optimizations

    - Sending one update per member wasn't a good idea (thousands of requests, could be my pc but better fix that)
    - A queue system to reduce the amount of requests
    - Using an interval and events to flush the queue regularly
    - Same issue wit versioning, now we clean all the extra versions for the full graph in an interval, which is way better than checking duplicates with every member
    - Using `jsonld-streaming-parser` to convert the jsonld into rdfjs
    - Using sparql js to make inserts from the rdfjs (so using a proper tool instead of parsing jsonld manually)
    D34DPlayer committed Aug 17, 2021
    Configuration menu
    Copy the full SHA
    7a2585f View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2021

  1. Configuration menu
    Copy the full SHA
    893887e View commit details
    Browse the repository at this point in the history
  2. Add version materialization as a new graphdb connector

    Main differences to the main one:
    - the quads will have the element as their subject, instead of the version
    - the `isVersionOf` field is replaced by a `hasVersion` (so we can always know what version is the one stored)
    D34DPlayer committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    9c9223f View commit details
    Browse the repository at this point in the history
  3. Cleanup before merge

    D34DPlayer committed Aug 18, 2021
    Configuration menu
    Copy the full SHA
    cce8788 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2021

  1. Add Lucene connector support

    Setting added to create a Lucene index automatically,
    this will allow for full text search on one field of the ldes
    D34DPlayer committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    dd0a966 View commit details
    Browse the repository at this point in the history