Skip to content

[gpu-v2026] Is job_execution_summary only supported from around Day 109? #259

Description

@Incinciblecolonel

Hi ASI Trace Team,

Thank you for releasing the Alibaba Cluster Trace GPU v2026 dataset.

We are using the trace for task-resolved workload modeling and trace-driven GPU cluster scheduling. During our audit, we observed a very clear temporal boundary when linking asi_opensource_pod_hourly with asi_opensource_job_execution_summary.

Using only pod-ID coverage, before applying any Duration threshold or matching score, we obtain:

  • Day 4–108: only about 0.2685% of reconstructed pod execution episodes have a pod ID that appears in job_execution_summary.
  • Day 109–129: the coverage becomes 100%.
  • Among our reliably linked execution episodes, 99.7845% come from Day 109 onward.
  • Most of the small number of matches before Day 109 are long-running episodes that continue across the Day-109 boundary.

This suggests that the temporal boundary exists in the released data itself, rather than being caused by our execution matching procedure.

We noticed that Issue #257 also asks about the missing submission timestamp in job_execution_summary, and Issue #258 reports that state_public is mostly Unknown during roughly the first 100 days. Our observation may be related to the same metadata-coverage boundary.

Could you please clarify:

  1. Does asi_opensource_job_execution_summary intentionally cover only executions from approximately Day 109 onward?
  2. If so, is there any execution-summary data for Day 0–108 that could potentially be released?
  3. Is there any anonymized field, mapping table, or execution-instance ID that can associate a row in job_execution_summary with a relative day / hour in pod_hourly?
  4. How exactly is duration_hours defined? In particular, does it represent wall-clock execution span, active allocated execution time, or another quantity?
  5. Would it be possible to release the upstream script used to construct job_execution_summary, or relative start/end timing information?

We do not need calendar dates, user identities, raw pod names, or any business-sensitive metadata. Relative day/hour information or a stable anonymized execution ID would be sufficient for our research.

We would be happy to share our audit scripts and aggregate linkage statistics if useful.

Thank you very much for releasing this valuable dataset.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions