Skip to content

Python web scraping tools for extracting eVTOL aircraft data and maintaining a dataframe.

Notifications You must be signed in to change notification settings

rovertm/eVTOL-Aircraft-Data-Extraction

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation

eVTOL Aircraft Data Extraction and Analysis

Web scraping tools for eVTOL aircraft data extraction maintaining a dataframe

Table of Contents

  1. About
  2. Use Cases
  3. Installation
  4. File Descriptions
  5. Licensing, Authors, and Acknowledgements
  6. View the notebooks

About

This repository contains tools for scraping eVTOL aircraft data from two websites:

  1. https://evtol.news/
  2. https://transportup.com/

Each set of scraper functions, contained in individual packages, follows an identical process architecture show below.

scraper_flow

Use Cases

Source data for:

  • Market research
  • Competitive intelligence

Installation

  • BeautifulSoup
  • Standard libraries across Python 3* Anaconda distribution.

File Descriptions

  • Jupyter Notebook for analysis:

    • evtolnews_scrape.ipynb
    • transportup_scrape.ipynb
  • Python files for custom functions

    • transportup_funcs.py
    • evtolnews_scrapefuncs.py
  • .csv files for read and writing to directory and results dataframes

    • tu_directory_{date}.csv --> dataframe of aircraft links (to scrape)

    • tu_results_df_{date}.csv --> dataframe of scraped and cleaned aircraft data

    • evtolnews_directory_{date}.csv --> dataframe of aircraft links (to scrape)

    • evtolnews_results_df_{date}.csv --> dataframe of scraped and cleaned aircraft data

Licensing, Authors, Acknowledgements

All code is open for any and all usage.

View the notebooks

View the evtol.news scraper at Jupyter's NBviewer site, click here.

View the TransportUP scraper at Jupyter's NBviewer site, click here.

About

Python web scraping tools for extracting eVTOL aircraft data and maintaining a dataframe.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published