Skip to content
View srujankgandla's full-sized avatar
0 followers · 1 following

Block or report srujankgandla

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
srujankgandla/README.md

Hi, I'm Srujan 👋

Data engineer with ~10 years of experience building large-scale data platforms on AWS and Azure — PySpark, Apache Iceberg lakehouses, and event-driven architectures. I contribute to open-source data tooling.

Open Source Contributions

OpenLineage: the open standard for data lineage

  • SSL context support for the Python HTTP transport: added an ssl_context option to HttpConfig so clients can use custom CA bundles and mTLS (issue #3460, PR #4983 — under review).

SQLMesh: data transformation framework

  • Fix: drop clustering key before dropping columns it references (issue #5813, PR #6095 — under review).

Dagster: data orchestration platform

  • Fix execution hang when a skip is unblocked by an abandon: plan_events_iterator now runs the skip/abandon phases to a fixed point instead of a single pass, so a skip unblocked by an abandonment can no longer stall execution forever (issue #33661, PR #34238 — under review).

PyIceberg: Apache Iceberg's Python library

  • Fix: stream record batches lazily in to_arrow_batch_reader(): the batch reader materialized every file up front (~12.6x memory blowup); it now streams batches one at a time (issue #2407, PR #4028 — under review).

Research

  • Silent data loss in serverless Spark — arXiv paper (2026).

Popular repositories Loading

  1. pypii-lineage pypii-lineage Public

    PyPIILineage: Automatic PII-aware column lineage propagation for Apache PySpark. Classifies output columns as Full PII, Derived PII, or PII-Free by analyzing Spark's logical query plan — no data ac…

    Python

  2. spark-lambda-fault-tolerance spark-lambda-fault-tolerance Public

    SafeWriter: Fault-tolerant Delta Lake and Apache Iceberg writes for Spark-on-AWS-Lambda. Eliminates silent data loss caused by Lambda timeout during two-phase commit.

    Python

  3. spark-connect-security spark-connect-security Public

    Python

  4. OpenLineage OpenLineage Public

    Forked from OpenLineage/OpenLineage

    An Open Standard for lineage metadata collection

    Java

  5. sqlmesh sqlmesh Public

    Forked from SQLMesh/sqlmesh

    Scalable and efficient data transformation framework - backwards compatible with dbt.

    Python

  6. dagster dagster Public

    Forked from dagster-io/dagster

    An orchestration platform for the development, production, and observation of data assets.

    Python