Goal
Reproduce the full 84-benchmark evaluation suite of the FTA / DACE paper —
Wang, Dillig & Singh, Synthesis of Data Completion Scripts using Finite Tree
Automata (OOPSLA 2017, arXiv:1707.01469) —
as runnable Aeon FTA programming-by-example tasks under
examples/synthesis/dace/, complementing the worked examples already there.
What already exists
The paper's concretely-specified examples (Fig. 1 running example + the five
§2 motivating examples) are reproduced and machine-verified via -s fta:
Covered by tests/dace_test.py::test_fta_pbe_completion. The DSL primitives live
in libraries/Dace.ae.
The gap
The paper evaluates DACE on 84 tasks but only the 6 above are individually
published (as footnoted StackOverflow URLs). The other ~78 are not available:
- Composition (from the evaluation section): 46 data imputation (R / Python
pandas), 32 spreadsheet computation (Excel / Google Sheets), 6 relational-DB
virtual columns.
- How they were collected: StackOverflow search for the keywords
"data imputation", "missing value", "missing data", "spreadsheet formula".
- Not enumerated in the paper or its appendices (A–D are scoring, soundness,
complexity, VSA-vs-FTA — no benchmark list).
Where the original artifact was searched for (came up empty)
So we know whom to ask and what is/isn't public:
- DACE is a Java tool — not open-sourced.
- github.com/xwangsd (Xinyu Wang, lead author) — no public repos.
- utopia-group (Işıl Dillig's group) — no DACE / data-completion repo.
- MIPL-group (Wang's Michigan lab) — only SQL-synthesis repos.
- No Zenodo / artifact archive (OOPSLA'17 predates routine PACMPL artifact archival).
Action for maintainer: email the authors (Xinyu Wang xwangsd@umich.edu,
Işıl Dillig, Rishabh Singh) to request the original 84 benchmarks and/or the DACE
source. If obtained, transcribe them directly; otherwise reconstruct (below).
Proposed approach (if the originals can't be obtained)
Reconstruct an equivalent suite by re-scraping StackOverflow with the paper's
documented keywords, then encode each as a pbe/*.ae task (a Column/Grid
global + @example rows + a ?hole function of the cell index), extending
libraries/Dace.ae with any missing navigation/aggregation primitives. Note
clearly that a re-scraped set is DACE-inspired, not the verbatim original.
Checklist
References
Goal
Reproduce the full 84-benchmark evaluation suite of the FTA / DACE paper —
Wang, Dillig & Singh, Synthesis of Data Completion Scripts using Finite Tree
Automata (OOPSLA 2017, arXiv:1707.01469) —
as runnable Aeon FTA programming-by-example tasks under
examples/synthesis/dace/, complementing the worked examples already there.What already exists
The paper's concretely-specified examples (Fig. 1 running example + the five
§2 motivating examples) are reproduced and machine-verified via
-s fta:MINUS(?1,?2)examples/synthesis/dace/pbe/delta.aepbe/locf.aepbe/prev_sameid.aepbe/turns.aepbe/group_count.aepbe/fallback.aeCovered by
tests/dace_test.py::test_fta_pbe_completion. The DSL primitives livein
libraries/Dace.ae.The gap
The paper evaluates DACE on 84 tasks but only the 6 above are individually
published (as footnoted StackOverflow URLs). The other ~78 are not available:
pandas), 32 spreadsheet computation (Excel / Google Sheets), 6 relational-DB
virtual columns.
"data imputation","missing value","missing data","spreadsheet formula".complexity, VSA-vs-FTA — no benchmark list).
Where the original artifact was searched for (came up empty)
So we know whom to ask and what is/isn't public:
Action for maintainer: email the authors (Xinyu Wang xwangsd@umich.edu,
Işıl Dillig, Rishabh Singh) to request the original 84 benchmarks and/or the DACE
source. If obtained, transcribe them directly; otherwise reconstruct (below).
Proposed approach (if the originals can't be obtained)
Reconstruct an equivalent suite by re-scraping StackOverflow with the paper's
documented keywords, then encode each as a
pbe/*.aetask (aColumn/Gridglobal +
@examplerows + a?holefunction of the cell index), extendinglibraries/Dace.aewith any missing navigation/aggregation primitives. Noteclearly that a re-scraped set is DACE-inspired, not the verbatim original.
Checklist
examples/synthesis/dace/libraries/Dace.aewith primitives each task needstests/dace_test.pyand updateexamples/synthesis/dace/README.mdTablecell extraction (GetCell navigation) rather than per-Columnindex functions, to match the paper's DSL more faithfullyReferences
examples/synthesis/dace/README.md,libraries/Dace.ae,tests/dace_test.py