Conference paper

HONG Chih-Duo, CHEN Yu-Fang, LENGÁL Ondřej, MU Shin-Cheng, SINHA Nishant and WANG Bow-Yaw. An Executable Sequential Specification for Spark Aggregation. In: Proceedings of NETYS'17. Heidelberg: Springer Verlag, 2017, pp. 1-15. ISSN 0302-9743.
Publication language:english
Original title:An Executable Sequential Specification for Spark Aggregation
Title (cs):Spustitelná sekvenční specifikace pro agregaci v prostředí Spark
Pages:1-15
Proceedings:Proceedings of NETYS'17
Conference:The 5th international conference on networked systems
Place:Heidelberg, DE
Year:2017
Journal:Lecture Notes in Computer Science, DE
ISSN:0302-9743
Publisher:Springer Verlag
Keywords
Data Parallel Computation, Functional Specification, Requirements, Verification, Spark
Annotation
Spark is a new promising platform for scalable data-parallel computation. It provides several high-level application programming interfaces (APIs) to perform parallel data aggregation. Since execution of parallel aggregation in Spark is inherently non-deterministic, a natural requirement for Spark programs is to give the same result for any execution on the same data set. We present PureSpark, an executable formal Haskell specification for Spark aggregate combinators. Our specification allows us to deduce the precise condition for deterministic outcomes from Spark aggregation. We report case studies analyzing deterministic outcomes and correctness of Spark programs.
BibTeX:
@INPROCEEDINGS{
   author = {Chih-Duo Hong and Yu-Fang Chen and Ond{\v{r}}ej Leng{\'{a}}l
	and Shin-Cheng Mu and Nishant Sinha and Bow-Yaw Wang},
   title = {An Executable Sequential Specification for Spark Aggregation},
   pages = {1--15},
   booktitle = {Proceedings of NETYS'17},
   journal = {Lecture Notes in Computer Science},
   year = {2017},
   location = {Heidelberg, DE},
   publisher = {Springer Verlag},
   ISSN = {0302-9743},
   language = {english},
   url = {http://www.fit.vutbr.cz/research/view_pub.php.en?id=11330}
}

Your IPv4 address: 54.224.50.28
Switch to IPv6 connection

DNSSEC [dnssec]