How to use the dagster.pipeline function in dagster

To help you get started, we’ve selected a few dagster examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github dagster-io / dagster / python_modules / dagster / dagster_tests / core_tests / test_composites.py View on Github external
    @pipeline
    def test_pipeline_double():
        diamond_composite.alias('D1')()
        diamond_composite.alias('D2')()
github dagster-io / dagster / python_modules / dagster / dagster_tests / core_tests / definitions_tests / test_composition.py View on Github external
    @pipeline
    def recurse():
        @composite_solid
        def outer():
            try:

                @composite_solid
                def throws():
                    called['throws'] = True
                    raise Garbage()

                throws()
            except Garbage:
                add_one(return_one())

        outer()
github dagster-io / dagster / examples / dagster_examples / intro_tutorial / composition.py View on Github external
@pipeline
def composed_pipeline():
    etl.alias('sales_etl')(sales_team_path())
    hr_table = etl.alias('hr_etl')(hr_team_path())
    analysis(hr_table)
github dagster-io / dagster / examples / dagster_examples / toys / fan_in_fan_out.py View on Github external
    @pipeline(name=name)
    def _pipe():

        return_one_out = return_one()
        prev_level_out = return_one_out
        for level in range(0, levels):
            prev_level_out = construct_fan_in_level(prev_level_out, level, fanout)