Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingest): support view lineage for all sqlalchemy sources #9039

Commits on Oct 19, 2023

  1. feat(ingest): support view lineage for all sqlalchemy sources

    Additional Changes:
    1. Support incremental lineage for all sqlalchemy sources
    2. Keep column level lineage enabled and incremental lineage disabled by default
    3. Monkey-patch hive dialect to extract hive view definitions to extract lineage
    4. Fix incremental_lineage_helper for empty upstreams
    
    Pending Followup Changes:
    1. Support postgres-like partial view definitions
    mayurinehate committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    75f63a2 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. revert incremental lineage helper refractor to source.py

    - primarily to reduce adverse effect on other sources, such as dbt
    which have their own flavour of incremental lineage implementation
    mayurinehate committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    93b7ba1 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Configuration menu
    Copy the full SHA
    7c18697 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Configuration menu
    Copy the full SHA
    bf4ecb5 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. feat(ingest): support view lineage extraction for postgres, trino

    fallback to native postgres view lineage extraction for failed views
    mayurinehate committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    b74e794 View commit details
    Browse the repository at this point in the history
  2. fix tests

    mayurinehate committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    337ca34 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e60a7f4 View commit details
    Browse the repository at this point in the history
  4. add sqlglot dependency

    mayurinehate committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    1fe4b02 View commit details
    Browse the repository at this point in the history
  5. update comment

    mayurinehate committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    c5ca9b2 View commit details
    Browse the repository at this point in the history