Blog post: Using Custom Python connectors with PyAirbyte

[bindipankhudi]

Summary

This blog post would focus on how to build custom python connectors and then executing them from PyAirbyte.

Definition of Done

Blog post illustrating how to build custom python connectors and then executing them from PyAirbyte.

Article should cover:

• Basic instruction on how to start building and testing a Python-based source connector.
• How to install and test a Python connector in PyAirbyte using pip_url of local path.
• Show example for installing a Python connector in PyAirbyte using pip_url of a git repo and/or with the pip_url of a git feature branch.
• Any suggestions for debugging / troubleshooting which seem relevant for this use case.

Resources to Assist

• PyAirbyte tutorials: https://github.com/airbytehq/quickstarts/tree/main/pyairbyte_notebooks
• PyAirbyte Documentaton: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started
• PyAirbyte API Reference: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started#api-reference
• Sample blog post: https://airbyte.com/blog/manage-and-orchestrate-airbyte-connections-using-python

[avirajsingh7]

@aaronsteers i think both this issue are almost same airbytehq/PyAirbyte-Hackathon#7

[aaronsteers]

@avirajsingh7 - Apologies for the confusion. I've updated this issue's description and title to target Python connectors, specifically, focusing on local testing and how to run from feature branches during development. I've also clarified the similar item to focus more clearly on no-code developer, which outputs the connector definition as a yaml file.

Let me know if this is still interesting - and if so, let me know if I can provide any additional resources to assist or clarify. Thanks!

[marcosmarxm]

@topefolorunso do you want to work on this one?

[Harmaton]

Assign this to me @marcosmarxm

[marcosmarxm]

@Harmaton do you need any help here?

[Harmaton]

@marcosmarxm I have this doc ready. Could you please review ? @bindipankhudi

[Harmaton]

If done. can you assign me issue #19 @marcosmarxm

[aaronsteers]

I completed a review and it looks good for my part. I'll let @marcosmarxm handle next steps during his working hours tomorrow. (Today is a US / company holiday.)

[Harmaton]

@marcosmarxm

[marcosmarxm]

@Harmaton you can proceed to the next challenge. I'll post this in Airbyte blog.

[Harmaton]

Please mark as done while at it. I know You have my bio. Thank you!