Databricks integration properties

[rdstn]

Hello,

I've been trying to run a Databricks configuration, but am turning up with an error. Am I missing something, or is it a genuine bug that I've hit?

The driver I'm running is the most recent, 2.6.32 version of the Databricks driver.
My configuration is, as follows:

jdbc.url=jdbc:spark://<databricksURL>:443/default;transportMode=http;ssl=1;httpPath=<path>;AuthMech=3;UID=token;PWD=<pass>
jdbc.driver=com.databricks.client.jdbc.Driver
com.databricks.client.jdbc.Driver-symbolFactory=it.unibz.inf.ontop.model.term.functionsymbol.db.impl.SparkSQLDBFunctionSymbolFactory
com.databricks.client.jdbc.Driver-typeFactory=it.unibz.inf.ontop.model.type.impl.SparkSQLDBTypeFactory
com.databricks.client.jdbc.Driver-serializer=it.unibz.inf.ontop.generation.serializer.impl.SparkSQLSelectFromWhereSerializer
com.databricks.client.jdbc.Driver-normalizer=it.unibz.inf.ontop.generation.normalization.impl.AlwaysProjectOrderByTermsNormalizer
com.databricks.client.jdbc.Driver-metadataProvider=it.unibz.inf.ontop.dbschema.impl.SparkSQLDBMetadataProvider

Then, the error I'm getting is:

Caused by: com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) [Guice/NullInjectedIntoNonNullable]: null returned by binding at JDBCMetadataProviderFactory.getMetadataProvider()
 but the 1st parameter of SparkSQLDBMetadataProvider.<init>(SparkSQLDBMetadataProvider.java:14) is not @Nullable
  at JDBCMetadataProviderFactory.getMetadataProvider(JDBCMetadataProviderFactory.java:1)
  at SparkSQLDBMetadataProvider.<init>(SparkSQLDBMetadataProvider.java:14)
      \_ for 1st parameter
  while locating DBMetadataProvider annotated with @UniqueAnnotations$Internal(value=1)

Learn more:
  https://github.com/google/guice/wiki/NULL_INJECTED_INTO_NON_NULLABLE

1 error

======================
Full classname legend:
======================
DBMetadataProvider:          "it.unibz.inf.ontop.dbschema.DBMetadataProvider"
JDBCMetadataProviderFactory: "it.unibz.inf.ontop.dbschema.impl.JDBCMetadataProviderFactory"
SparkSQLDBMetadataProvider:  "it.unibz.inf.ontop.dbschema.impl.SparkSQLDBMetadataProvider"
UniqueAnnotations$Internal:  "com.google.inject.internal.UniqueAnnotations$Internal"
========================
End of classname legend:
========================

        at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:251)
        at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1104)
        at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:906)
        at com.sun.proxy.$Proxy139.getMetadataProvider(Unknown Source)
        at it.unibz.inf.ontop.spec.mapping.impl.SQLMappingExtractor.convert(SQLMappingExtractor.java:217)
        at it.unibz.inf.ontop.spec.mapping.impl.SQLMappingExtractor.convert(SQLMappingExtractor.java:185)
        at it.unibz.inf.ontop.spec.mapping.impl.SQLMappingExtractor.convertPPMapping(SQLMappingExtractor.java:151)
        at it.unibz.inf.ontop.spec.mapping.impl.SQLMappingExtractor.extract(SQLMappingExtractor.java:106)
        at it.unibz.inf.ontop.spec.impl.DefaultOBDASpecificationExtractor.extract(DefaultOBDASpecificationExtractor.java:46)
        at it.unibz.inf.ontop.injection.impl.OntopMappingConfigurationImpl.loadSpecification(OntopMappingConfigurationImpl.java:127)
        at it.unibz.inf.ontop.injection.impl.OntopMappingSQLConfigurationImpl.loadSpecification(OntopMappingSQLConfigurationImpl.java:87)
        at it.unibz.inf.ontop.injection.impl.OntopMappingSQLAllConfigurationImpl.loadSpecification(OntopMappingSQLAllConfigurationImpl.java:47)
        at it.unibz.inf.ontop.injection.impl.OntopSQLOWLAPIConfigurationImpl.loadOBDASpecification(OntopSQLOWLAPIConfigurationImpl.java:34)
        at it.unibz.inf.ontop.injection.impl.OntopOBDAConfigurationImpl.loadSpecification(OntopOBDAConfigurationImpl.java:44)
        at it.unibz.inf.ontop.injection.OntopSystemConfiguration.loadQueryEngine(OntopSystemConfiguration.java:20)
        at it.unibz.inf.ontop.rdf4j.repository.impl.OntopVirtualRepository.initializeInternal(OntopVirtualRepository.java:72)
        ... 10 common frames omitted

Looks to me like the system's not able to pick up the metadata provider.

[bcogrel]

Hi @rdstn,

Which version of Ontop are you using? We officially support Databricks since 5.0.0 and already have these entries:
https://github.com/ontop/ontop/blob/version5/db/rdb/src/main/resources/it/unibz/inf/ontop/injection/sql-default.properties#L123

4.2.2 already supported Apache Spark, but we had to do several fixes to make it work with Databricks.

For recent versions of Databricks's JDBC driver, we use jdbc:databricks:// as prefix. The httpPath contains the path to the data warehouse.

As for the user and the password, we usually pass them using the entries jdbc.user and jdbc.password.

Best,
Benjamin

[rdstn]

A-ha! It was the jdbc:databricks vs jdbc:spark. I tried with 5.0.2 and 4.2.0 - hence the explicit setting of the metadata provider, etc., for the earlier version.