cnolanminich

# ---------------------------------------------------------------------------
# 1. Upstream asset — plain @asset
# ---------------------------------------------------------------------------

@dg.asset(
    group_name="pipeline",
    kinds={"python"},
    tags={"domain": "orders"},
)
def raw_orders(context: dg.OpExecutionContext) -> list[dict]:

cnolanminich

#!/usr/bin/env python3
#!/usr/bin/env python3
# /// script
# dependencies = [
#   "requests<3",
# ]
# ///
import argparse
import sys
import requests

cnolanminich

Asset Granularity Demo

This demo project showcases three different approaches to modeling ETL operations as Dagster assets, each with different trade-offs for visibility, control, and simplicity.

Quick Start

# Activate virtual environment
source .venv/bin/activate

cnolanminich

The Two-Step Approach to Dynamic Dagster+ Deployments

Creating a new persistent deployment requires coordinating two systems: the Dagster+ API and your Kubernetes agent configuration.

1. Dagster+ API (Create the Deployment Resource)

The deployment must exist in Dagster+ as a logical entity. This is done via the GraphQL API:

import httpx

cnolanminich

Per-Asset Concurrency with Dagster Pools

This document explains how to limit concurrent executions of individual assets using Dagster's pool-based concurrency system.

Overview

By default, Dagster allows unlimited concurrent executions of the same asset across multiple runs. This can cause issues when:

• An asset writes to a shared resource (database table, file, API)
• An asset has rate limits or resource constraints
• You want to prevent duplicate work

cnolanminich

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "requests",
# ]
# ///
"""
Query Dagster's GraphQL API to get monthly credits (ops executed + asset materializations).

cnolanminich

Sigma Data Models Implementation Plan

Executive Summary

This document outlines a detailed plan to add support for Sigma's new Data Models API to the dagster-sigma library. Data Models are replacing the deprecated datasets (end-of-life: June 2, 2026) and provide enhanced semantic layer capabilities including metrics, relationships, column/row-level security, and more.

Timeline: ~5-8 days development + testing Priority: Medium-High (18 month deprecation timeline for datasets) Complexity: Moderate

cnolanminich

mutation {
  createSecret(
    secretName: "MY_API_KEY"
    secretValue: "abc123"
    scopes: { fullDeploymentScope: true }
    locationNames: ["prod"]
  ) {
    __typename
    ... on Error {
      message

cnolanminich

Dagster Demo Generator - GitHub Copilot Guide

This document provides comprehensive instructions for GitHub Copilot to replicate the Dagster demo generation workflow implemented in this repository.

Architecture Overview

The Dagster Demo Generator automates the creation of demonstration Dagster projects using a multi-tiered workflow:

1. Command Entry Point: /create-dagster-demo slash command (.claude/commands/create-dagster-demo-2.md)
2. Skills System: Three specialized skills that handle different aspects of Dagster project creation

cnolanminich

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
from airflow.settings import Session
from airflow.models import DagRun
from datetime import timedelta
import pandas as pd
import numpy as np

# Constants for flexibility