Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lineage to ingest SQL Server SSIS packages #16296

Open
Jigawatter opened this issue May 16, 2024 · 0 comments
Open

Lineage to ingest SQL Server SSIS packages #16296

Jigawatter opened this issue May 16, 2024 · 0 comments
Assignees
Labels

Comments

@Jigawatter
Copy link

Problem:
We use SQL Server SSIS packages heavily to extract/transform/load data between multiple servers and sources. It's very difficult and frustrating to keep track of these packages and manage dependencies.
Users often ask "What is the source of this data?" and it can be difficult for us to answer confidently without significant effort, due to these packages not being mapped.
Additionally, some of these packages contain business logic that is difficult to access without opening each package individually.

OMD currently shows the lineage of database objects, but mapping across databases and servers via these packages is not possible.

Solution:
I would like to see ingestion workflows for SQL Server SSIS that can extend the existing lineage for SQL Server (and other sources/destinations), such that in the Lineage view for a data asset i can see that it feeds to or from an SSIS package and then onto another data asset elsewhere in our environment.

Alternatives:
We've tried to manually document in a spreadsheet all the SSIS packages we have, but it was challenging to maintain and quickly drifted out of date.
Currently we are investigating what data can be automatically extracted via scripts from the SSIS xml files or the SSIS catalog db - maybe then we can import that into OMD and manually update the lineage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants