Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add firebird query runner #6730

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 14 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ RUN if [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg \
&& curl https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql17 \
&& ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql17 libtommath1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& curl "$databricks_odbc_driver_url" --location --output /tmp/simba_odbc.zip \
Expand All @@ -76,7 +76,19 @@ RUN if [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
&& dpkg -i /tmp/simba/*.deb \
&& printf "[Simba]\nDriver = /opt/simba/spark/lib/64/libsparkodbc_sb64.so" >> /etc/odbcinst.ini \
&& rm /tmp/simba_odbc.zip \
&& rm -rf /tmp/simba; fi
&& rm -rf /tmp/simba; fi \
&& curl "https://github.com/FirebirdSQL/firebird/releases/download/v5.0.0/Firebird-5.0.0.1306-0-linux-x64.tar.gz" --location --output /tmp/firebird.tar.gz \
&& mkdir /tmp/firebird \
&& tar -xvf /tmp/firebird.tar.gz --directory /tmp/firebird \
&& mkdir /tmp/firebird/buildroot \
&& tar -xvf /tmp/firebird/Firebird-*/buildroot.tar.gz --directory /tmp/firebird/buildroot \
&& cp /tmp/firebird/buildroot/opt/firebird/lib/libfbclient.so /usr/lib/libfbclient.so.2 \
&& ln -s /usr/lib/libfbclient.so.2 /usr/lib/libfbclient.so \
&& cp /tmp/firebird/buildroot/opt/firebird/lib/libtomcrypt.so /usr/lib/libtomcrypt.so.1 \
&& ln -s /usr/lib/libtomcrypt.so.1 /usr/lib/libtomcrypt.so \
&& rm /tmp/firebird.tar.gz \
&& rm -rf /tmp/firebird \
&& ln -s /usr/lib/x86_64-linux-gnu/libtommath.so.1 /usr/lib/x86_64-linux-gnu/libtommath.so.0
Comment on lines +79 to +91
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this really need to be installed for a firebird connector to work? I'd imagine we can just ping an existing firebird db to query it - I don't think we'd have to actually install it in the redash image.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we need some client libraries to be able to connect to a Firebird database.
See the docs for more info.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks.


WORKDIR /app

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Redash supports more than 35 SQL and NoSQL [data sources](https://redash.io/help
- Elasticsearch
- Exasol
- Microsoft Excel
- Firebird
- Firebolt
- Databend
- Google Analytics
Expand Down
Binary file added client/app/assets/images/db-logos/firebird.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 15 additions & 1 deletion poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ werkzeug = "2.3.8"
wtforms = "2.2.1"
xlsxwriter = "1.2.2"
tzlocal = "4.3.1"
fdb = "^2.0.2"

[tool.poetry.group.all_ds]
optional = true
Expand Down
152 changes: 152 additions & 0 deletions redash/query_runner/firebird.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
import datetime
import logging

from redash.query_runner import (
TYPE_DATE,
TYPE_DATETIME,
TYPE_FLOAT,
TYPE_INTEGER,
TYPE_STRING,
BaseSQLQueryRunner,
JobTimeoutException,
register,
)

logger = logging.getLogger(__name__)

try:
import fdb

enabled = True
except ImportError:
enabled = False

Check warning on line 22 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L21-L22

Added lines #L21 - L22 were not covered by tests

types_map = {
str: TYPE_STRING,
int: TYPE_INTEGER,
float: TYPE_FLOAT,
datetime.date: TYPE_DATE,
datetime.datetime: TYPE_DATETIME,
}


class firebird(BaseSQLQueryRunner):
noop_query = "SELECT 1 FROM RDB$DATABASE;"

limit_query = " FIRST 1000"
limit_keywords = ["FIRST"]
limit_after_select = True

@classmethod
def configuration_schema(cls):
return {
"type": "object",
"properties": {
"user": {"type": "string"},
"password": {"type": "string"},
"server": {"type": "string", "default": "127.0.0.1"},
"port": {"type": "number", "default": 3050},
"charset": {
"type": "string",
"default": "UTF-8",
"title": "Character Set",
},
"db": {"type": "string", "title": "Database Name"},
},
"required": ["db"],
"secret": ["password"],
}

@classmethod
def enabled(cls):
return enabled

@classmethod
def name(cls):
return "Firebird"

@classmethod
def type(cls):
return "firebird"

def _get_tables(self, schema):
query = """

Check warning on line 73 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L73

Added line #L73 was not covered by tests
SELECT TRIM(f.rdb$relation_name) AS table_name
,TRIM(f.rdb$field_name) AS column_name
FROM rdb$relation_fields f
JOIN rdb$relations r ON f.rdb$relation_name = r.rdb$relation_name
AND r.rdb$view_blr IS NULL
AND (r.rdb$system_flag IS NULL OR r.rdb$system_flag = 0)
ORDER BY 1, f.rdb$field_position;
"""

results, error = self.run_query(query, None)

Check warning on line 83 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L83

Added line #L83 was not covered by tests

if error is not None:
self._handle_run_query_error(error)

Check warning on line 86 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L86

Added line #L86 was not covered by tests

for row in results["rows"]:
table_name = row["TABLE_NAME"]

Check warning on line 89 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L89

Added line #L89 was not covered by tests

if table_name not in schema:
schema[table_name] = {"name": table_name, "columns": []}

Check warning on line 92 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L92

Added line #L92 was not covered by tests

schema[table_name]["columns"].append(row["COLUMN_NAME"])

Check warning on line 94 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L94

Added line #L94 was not covered by tests

return list(schema.values())

Check warning on line 96 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L96

Added line #L96 was not covered by tests

def run_query(self, query, user):
connection = None

Check warning on line 99 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L99

Added line #L99 was not covered by tests

try:
server = self.configuration.get("server", "")
user = self.configuration.get("user", "")
password = self.configuration.get("password", "")
db = self.configuration["db"]
port = self.configuration.get("port", 3050)
charset = self.configuration.get("charset", "UTF-8")

Check warning on line 107 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L101-L107

Added lines #L101 - L107 were not covered by tests

if port != 3050:
server = server + "/" + str(port)

Check warning on line 110 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L110

Added line #L110 was not covered by tests

connection = fdb.connect(

Check warning on line 112 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L112

Added line #L112 was not covered by tests
database=f"{server}:{db}",
user=user,
password=password,
charset=charset,
)

if isinstance(query, str):
query = query.encode(charset)

Check warning on line 120 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L120

Added line #L120 was not covered by tests

cursor = connection.cursor()

Check warning on line 122 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L122

Added line #L122 was not covered by tests

cursor.execute(query)
data = cursor.fetchall()

Check warning on line 125 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L124-L125

Added lines #L124 - L125 were not covered by tests

if cursor.description is not None:
columns = self.fetch_columns([(i[0], types_map.get(i[1], None)) for i in cursor.description])
rows = [dict(zip((column["name"] for column in columns), row)) for row in data]

data = {"columns": columns, "rows": rows}
error = None

Check warning on line 132 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L131-L132

Added lines #L131 - L132 were not covered by tests
else:
error = "No data was returned."
data = None

Check warning on line 135 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L134-L135

Added lines #L134 - L135 were not covered by tests

cursor.close()
connection.commit()

Check warning on line 138 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L137-L138

Added lines #L137 - L138 were not covered by tests
except fdb.Error as e:
error = e.args[0]
data = None
except (KeyboardInterrupt, JobTimeoutException):
connection.cancel()
raise

Check warning on line 144 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L140-L144

Added lines #L140 - L144 were not covered by tests
finally:
if connection:
connection.close()

return data, error

Check warning on line 149 in redash/query_runner/firebird.py

View check run for this annotation

Codecov / codecov/patch

redash/query_runner/firebird.py#L149

Added line #L149 was not covered by tests


register(firebird)
1 change: 1 addition & 0 deletions redash/settings/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ def email_server_is_configured():
"redash.query_runner.ignite",
"redash.query_runner.oracle",
"redash.query_runner.e6data",
"redash.query_runner.firebird",
]

enabled_query_runners = array_from_string(
Expand Down