In astroinformatics, the processing and analyzing. Scaling Astroinformatics: Python + Automatic Parallelization COVER FEATURE

Transcription

1 Scaling Astroinormatics: Python + Automatic Parallelization Stean C. Müller, ETH Zürich and University o Applied Sciences Northwestern Switzerland Gustavo Alonso, ETH Zürich André Csillaghy, University o Applied Sciences Northwestern Switzerland Pydron, a system that bridges the gap between interactive data analysis and scalable computing, enables astronomers to process large datasets with Python without the compleities o managing parallel processing code. In astroinormatics, the processing and analyzing o astronomical data is inherently eploratory, and to derive useul insights rom the data, scientists must oten run many processing iterations. The time between deciding which processing to apply and obtaining results can limit both worklow eiciency and knowledge acquisition. As data volumes reach the eabyte scale, analyzing data at workstations becomes impossible. To obtain meaningul results rom that data, the partnership o science and scalable computing must be mutually beneicial. Unortunately, the trend toward big data is straining that partnership. Although computing platorms are now less epensive and scale to thousands o cores, they are also more comple, which creates steep learning curves or scientists. Such learning curves are a common problem across the IT industry, but their eect is acute in astroinormatics because data processing is oten tailored to a particular research line within a group, with only a ew hundred scientists actually working on a given dataset and processing results. Thus, as platorms advance, worklow becomes less eicient because scientists cannot evolve their data processing and programming practices to keep pace with new processing techniques and hardware advances. Changes are happening too rapidly to allow the needed economies o scale. The cycle o narrower dataset scope, higher platorm compleity, and increased available data is accelerating. In light o this cycle, rather than proposing new languages or parallel programming platorms, computer scientists need to develop immediately practical tools that provide a leible interace between the application domain and parallel computing. Scientists can then perorm their research independently without having to worry about modiying the parallel computing code. In collaboration with several astronomy groups, we have developed such a tool. By inserting just two decorators in sequential Python code, application developers can use Pydron ( to automatiy parallelize their code with a transparent and simple-to-use interace between the application and parallel computing domains. Although Pydron s underlying concepts could work with other languages, we chose Python because its powerul libraries, such as SciPy, 1 make it popular among astronomers. Our goal or Pydron is to let scientists work directly on large datasets as conveniently as they analyze data on a laptop. To evaluate Pydron s eectiveness, we eplored several use cases, rom cosmology to detecting planets outside the solar system, and eamined its role in a data- reduction pipeline. In all the use cases, we proved that Pydron requires minimal code changes. In /14/$ IEEE Published by the IEEE Computer Society SEPTEMBER

2 @schedule de analyze(eposures) = [] or e in eposures: += measure(e) de measure(eposure):... Pydron Figure 1. Regular, sequential Python code with two types o decorators that Pydron eecutes transparently in parallel. In this eample, Pydron knows that it should parallelize the code ollowing decorator. decorator tells Pydron that the measure unction has no side eects. the data-reduction eperiment, Pydron scaled to many CPU cores and reduced eecution time rom more than our hours to less than three minutes a 100-plus reduction actor. HOW PYDRON WORKS As Figure 1 shows, Pydron uses two decorators to automatiy parallelize regular, sequential Python code and distribute it to multiple cores, cluster nodes, or the cloud. It then sends results, such as plots, back to the astronomer s machine or inspection or urther processing. Decorators Developers insert two decorators in the Python marks unctions that contain the code or automatic parallelization, tells Pydron that a given unction is ree o side eects (it neither modiies state nor has any observable interaction with the outside world). Pydron parallelizes only unctions with decorator. I the available CPU cores are ully utilized, Pydron can automatiy decide not to analyze a unction with the decorator and instead treat it as a regular unction. Support scope Pydron takes care o scheduling jobs to the cluster or starting cloud nodes. It also transers code and entire Python libraries rom the scientist s computer to the worker nodes in the cluster or cloud. Applications launch as they usually do, such as rom within an integrated development environment. Unlike previous systems that operate on Python code, Pydron supports almost all Python language eatures, including nested unctions and lambda epressions. Consequently, the learning curve is shorter. Pydron is also easier to use on eisting code, since the developer need not stay within a limited eature set. The only restrictions are the yield statement and the lack o support or eception handling, which we are planning to include in Pydron s net release. Detecting parallelism and side eects Pydron can detect both task parallelism, in which dierent operations eecute independently, and data parallelism, in which a loop perorms the same operations on multiple data. Pydron can eecute two Python epressions or statements in parallel i both are ree o side eects and i there is no potential inormation low rom one to the other. Side-eect sources include changing o global variables (reading is not a side eect), in-place object modiication, interactions with the environment, and invoking unctions with side eects. The current Pydron version does not check these conditions, although we epect the net version to have methods that semiautomate this process. Developers indicate that a unction has no side eects by applying decorator. Many developers o scientiic applications have a background in mathematics, so a unctional sotware design might be more intuitive than one based on mutable objects and global state. Python s pickle API serializes objects or transport; pickle is designed or leibility, not raw speed, so passing large objects as parameters and return values can become a bottleneck. Pydron assumes that the application uses a separate communication channel, such as a shared ile system, to transer large data volumes between tasks. Developers must represent each ile with a Python object, such as a string containing the ile s name. I the application passes around such an object as a ile placeholder, reading and writing the ile is not a side eect. SAMPLE USE CASES To evaluate Pydron s eectiveness, we tested it on multiple use cases that cover several astronomical areas. Three cases show how Pydron parallelizes new code, improves eisting code, and helps data reduction. Enabling parallelism on eisting code To evaluate Pydron s perormance on eisting code, we used parameter-itting code or etra galactic 42 COMPUTER

3 simulation, which is in development at ETH Zürich s Institute o Astronomy. The code optimizes the parameters o wide-ield simulation sotware 2 to match real eposures o space outside the Milky Way, with thousands o galaies per eposure. A chisquare test uses distribution estimates based on binning to evaluate goodness-o-it, and parallel random search optimizations determine the parameter itting. The processing code is in Python. To harness a cluster s perormance potential, the developers were orced to split the code across several Python applications. A worklow system could orchestrate the splitting and application eecution, but the developers opted to write a simple bash script that interacts with the cluster s queuing system to perorm those tasks. The bash script, which contains the outermost process loops, is hard to maintain and speciic to one cluster. To keep its size manageable, the developers had to limit what they could parallelize. We removed the manually parallelized bash script and went back to a simple, sequential, Python script that we then parallelized with Pydron. Replacing the bash script had several immediate advantages. All the code was in one language, and in the process o revisiting the code, we identiied several additional unctions that Pydron could parallelize to improve scalability. Code changes were minor, such as speciying which iles to read and write inside some o the methods, so that Pydron could track communication between methods. These code changes also improved code readability. Improving parallelism PynPoint is a Python image-processing package to detect planets outside the solar system that orbit a single star. 3 The eoplanet s brightness is less than a millionth o the star s light, and both optical eects and atmospheric distortion spread the detected light over an area larger than the planet s orbit. PynPoint uses principal component analysis to model each star s point spread unction, removing the star s light but leaving the planet s light detectable in the residue. Because an eoplanet is so aint, astronomers must combine thousands o eposures to detect it. Several parts o PynPoint s detection algorithm use SciPy with multithread support, including the principal component analysis. The dataset is so large that shared-memory parallelization takes hours to process a single star. PynPoint users can tune several parameters, some o which aect only the last ew processing steps such as the number o principal components. Another last step is to rotate individual eposures to compensate or Earth s rotation. The rotation operation, which is encapsulated in a utility method, is not parallelized and runs with a single thread. Because it is a inal step in the algorithm, its lack o scalability is noticeable when tuning parameters By applying decorator on the code, Pydron parallelized the rotation operation with minimal eort. The loop over the individual images was part o a larger unction that was otherwise suiciently parallel, so we moved the loop into a separate unction and marked only this unction with decorator. We also replaced a modiication inside the loop with equivalent code that stored the rotated image as a new object. Data reduction A common step in pipelines to reduce astronomical data, particularly in surveys, is detecting and measuring light sources in images. Applying this process can involve thousands or even millions o images, so scalability is highly desirable. To demonstrate Pydron s ability to handle scale, we applied it in detecting and measuring sources in 576 images rom the Hubble Space Telescope, 4 each approimately 250 Mbytes. The source_etraction routine in Figure 2a etracts the light sources rom all images and concatenates the resulting catalogs (~180 Kbytes per image) into a single catalog ile. decorator instructs Pydron to parallelize this routine. source_etraction s the etract unction (not shown), which is decorated For each image, this unction invokes an eternal source etraction tool. 5 Etracting an eposure takes about 30 seconds, and the concatenate unction, also builds the inal catalog in about 40 seconds. Figure 2b shows concatenation and eecution times. For the irst 16 cores, scalability is with a single multicore node; thereater, it is with multiple nodes. A separate node represents the scientist s workstation, where Pydron perorms scheduling and graph rewriting. Eecution time scales almost linearly up to about 128 cores, ater which it lattens. This trend is not surprising, since some cores must process one more eposure than the others. For eample, with 512 cores, 64 cores will have to process two eposures (576 total). While those cores process their second eposure, the remaining cores are idle. Once all etractions have inished, the concatenation runs on a single core. The scheduling and graph rewriting code, which is not optimized, requires approimately 80 seconds o CPU time, independent o the core count. The code runs partially in parallel with the worker nodes, reducing the eective overhead to less than 30 seconds. Other Python constructs, such as while-loops, listcomprehension, or individual etract s per image instead o a loop, produce the same results with only minor changes. PARALLELIZATION GRANULARITY Parallelization requires splitting a process into several tasks, which deines the granularity. The iner the SEPTEMBER

4 @schedule de source_etraction(eposures): catalogs = [] or eposure in eposures: catalog = etract(eposure) catalogs += [catalog] return concatenate(catalogs) (a) 17 h 4 min 8 h 32 min 4 h 16 min 2 h 8 min Eecution time 1 h 4 min Concatenation time 32 min 16 min 8 min 4 min 2 min 1 min 30 s 15 s 7.5 s (b) CPU cores Time Figure 2. How Pydron improves scalability. (a) Python code with decorator, which tells Pydron to parallelize the code in source_etraction, and (b) resulting concatenation and eecution times as cores increase. The dotted line depicts the Python code s eecution time without Pydron. granularity, the simpler each individual task becomes. Automatic parallelization involves deciding when to run tasks in parallel, which is easier when the tasks are simple. However, the iner the granularity, the more overhead accrues rom orchestrating the split and managing task communication. Thus, parallelization and granularity must have the right balance. Fine-grained parallelization At a very ine granular level, the processor splits machine operations into micro-operations, which run in dierent pipeline stages. Processors perorm this parallelization automatiy or arbitrary code, but conditional branches limit the approach s scalability to a ew stages. At a slightly coarser granularity, the process split might produce tasks that can run on hundreds o cores o a graphical processing unit, or on multiple processors with shared memory. At this granularity, automatic parallelization o arbitrary code in arbitrary languages is no longer possible. SEJITS 6 and Parakeet 7 have demonstrated that, under restrictions, ine-grained parallelization on shared-memory inrastructure is still possible with Python. However, only a subset o the language is supported, and the developer must ollow restrictions to avoid side eects. Coarse-grained parallelization For systems that target inrastructure with multiple machines, such as a cluster or cloud, parallelization granularity is generally coarse. Systems that automatiy parallelize at this granularity, such as Swit, 8 require languages designed or automatic parallelization. These new languages introduce a learning curve. Having separate languages or parallelization also enorces a strict separation between orchestration and processing. Where the separation occurs deines how well the process will scale, but rom a conceptual or sotware design view, the demarcation is not always convenient. The same argument holds or worklow systems such as Taverna, 9 in which developers speciy a worklow graph in a graphical instead o a tetual programming language. Pydron targets the same computational inrastructure but parallelizes Python code, which shortens the user s learning curve. With one language, the barrier between orchestration and processing is less distinct. Pydron can oten automatiy reduce granularity until it suiciently parallelizes the code to scale to all available machines. Developers control the granularity level according to where they place decorator. They no longer have to make the big-picture granularity decision, and the code is less dependent on the inrastructure. PYTHON-TO-DATAFLOW TRANSLATION Pydron translates unctions marked into a datalow graph, which is directed, acyclic, and bipartite. Each epression and statement has a corresponding part on the graph, and nodes are either value-nodes or task-nodes. Each value-node represents a speciic value that becomes eplicit as Pydron evaluates the graph. The data that a value-node represents never changes. Task-nodes represent operations on values, the instructions or how 44 COMPUTER

5 to calculate output values rom input values. In a broader contet, value-nodes (,y) = (a,b) represent variables and tasknodes represent Python epressions and statements. Translation starts when (a) (b) Python s parser translates Python code into an abstract synta tree (AST), which Pydron traverses depth-irst. In the Figure 3 eample, the variables a, b,,, and y translate to value-nodes in the corresponding datalow graph, which represents the content o each variable at that eecution point. Pydron translates the unction to a task-node that perorms a operation. In the code in Figure 3a, is a regular variable that contains the ed unction. Because unctions in Python are objects, this object becomes input to the task. Pydron translates the s return value to an intermediate value-node,, with no corresponding variable. Such intermediate value-nodes behave in the same way as regular value-nodes. Python supports righthand-side (RHS) epressions, so the unpack epression translates to a task-node. Static single-assignment orm The static single-assignment orm 10 is a way to represent the values that each variable represents. Because a valuenode represents eactly one value or a variable, Pydron translates each variable into a series o value-nodes, with each value-node representing the value stored or a speciic time. When a statement assigns a variable (RHS epression), Pydron creates a new value-node that represents the variable s new value. When an epression reads a variable (lethand-side epression), Pydron uses the most recent value-node or that variable. Dynamic datalow reinement Translation using only the code has severe limitations. For eample, we must assume that most unction s have side eects, since it is generally not possible to identiy the ed unction rom the code alone. The resulting datalow graph contains edges that eectively turn a into a synchronization point. Python is a dynamic language, so these static analysis limitations are not surprising. Pydron counters these limitations with a dynamic datalow. During datalow evaluation, Pydron gains more inormation as it discovers the data that value-nodes represent. Pydron passes this inormation to the tasks, which can choose to use it at runtime to change the datalow graph in their neighborhood. Once it knows all the inputs or a particular task, Pydron eecutes the task. a b Figure 3. Translating (a) Python code into (b) a datalow graph. unpack Call task handling illustrates how a graph can change at runtime. Pydron inorms tasks as soon as it knows the ed unction. The task then checks or decorator. I it is present, the task can remove the edges used to model the side eects. I the ed method has decorator, the task will merge a copy o the ee s datalow into the main datalow unless the scheduler decides that that no urther reinement is required. Parts o the merged datalow might be able to start processing beore all arguments are known. Loop statements The dynamic datalow graph can also implement loops. Pydron translates the loop s body and else section as a subgraph and inserts a single task-node that represents the complete loop statement into the main graph. I the loop is a or-loop, one task input is the iterator. As soon as the iterator is available, Pydron unrolls the loop by merging copies o the body graph into the main graph. Because etching elements rom the iterator itsel might be an epensive operation or have side eects, Pydron unrolls the iterations in a tail-recursive pattern. I no side eects get in the way, Pydron can unroll the complete loop immediately. While-loops are handled similarly. The or-loop in Figure 4 shows the code and corresponding graph created during translation. Inside the body subgraph is another or-loop task that will orm the tail- recursive pattern. Although the igure does not depict them, the nested or task has the same two subgraphs as those in Figure 4b. Figure 5 shows the graph ater the iterator has become available and Pydron has unrolled the or-loop twice. The irst copy o the body graph has no or statement since the second copy replaced it. I the s are more epensive than the summation, Pydron can achieve good parallelism without making any aggregate assumptions on the += operator, which might not hold, since Python supports operator overloading. Return, break, and continue statements Because return, break, and continue statements interrupt the program s regular control low, they need a y SEPTEMBER

6 list or body s or in list: s += () iter + s s it s it or s else (a) (b) Figure 4. Loop translation in Pydron. (a) or-loop statement in Python and (b) the resulting datalow graph. The body subgraph has an input that Pydron will connect to the target during unrolling. The or-loop task has output, which corresponds to the target o the last iteration. s s + s s with the main process and between the nodes, and or eecuting queued tasks. The back end is modular and conigurable, and developers can implement additional back ends through a simple API. Scientists or their system administrators create each coniguration proile only once, and it will work or any Pydron application. it Figure 5. Partially unrolled or-loop statement. Pydron can eecute () in parallel even though the iterations are not independent because summation (s) must happen sequentially. special translation process. Pydron replaces these statements with lag variables that indicate a control-low interruption and with i blocks that tell it to skip the code ollowing the interrupt statement. Loop tasks are aware o these lag variables and use them to decide i more iterations should be unrolled and i the iteration should end with the else section. BACK-END SUPPORT Datalow evaluation is responsible or creating a queue o tasks that need eecuting. The back end is responsible or starting worker nodes, establishing communication or Cluster and cloud When the application starts, the back end connects to the cluster and requests a number o nodes by opening a Secure Shell (SSH) connection to the cluster s log-in node and eecuting the necessary commands. Commands are ully conigurable. In Pydron s current implementation, nodes keep running until the program inishes, which can entail sending multiple tasks to each node or eecution. The worker nodes start the Python interpreter, which we assume is installed there. A tiny bootstrap script starts a multiphase initialization procedure to transer the Pydron ramework to the worker node. Pydron communicates with the worker nodes through a TCP/IP connection. I irewalls block a direct connection, it routes communication through the log-in node and SSH connection. 46 COMPUTER

7 Pydron then places an import hook in the worker node s Python interpreter, which loads Python modules through the network i a module is not installed on the worker. This step eliminates the need or developers to manually keep an updated version o their projects on each worker node. Cloud support is similar to cluster support, ecept that Pydron irst boots the nodes through a cloud API, such as Apache libcloud ( In Pydron s current implementation, nodes shut down once the program has inished, but our plan is to have more elaborate node management. Multicore support The multicore back end eecutes tasks on the local machine, but in multiple processes that harness the power o all cores. The shared memory on such a machine would enable highly eicient communication. However, Pydron is designed primarily or use on inrastructures and applications without shared memory, which do not require ine-grained communication. Pydron eploits a high-perormance computing inrastructure rom regular, sequential Python code. Our use cases have demonstrated the need or Pydron in real-world applications with methods that can be parallelized at a coarse granularity. Scientists who have tested Pydron have commented on its ease o use and short learning curve. We are working on support or the yield statement and eception handling, which will give Pydron ull support o the Python language. We also plan to release Pydron in irst quarter 2015 under an open source license. We hope that others will see a key lesson in our work: reconciling science and scalable computing is less about optimal inrastructure use and more about simplicity. The main goal should be allowing scientists to ocus on their research instead o on data analysis technology. Reerences 1. E. Jones et al., SciPy: Open Source Scientiic Tools or Python, 2001; 2. J. Bergé et al., An Ultraast Image Generator (UIG) or Wide-Field Astronomy, Astronomy and Computing, vol. 1, Feb. 2013; S A. Amara and S.P. Quanz, PynPoint: An Image Processing Package or Finding Eoplanets, Monthly Notices o the Royal Astronomical Society, vol. 427, no. 2, 2012, pp N. Scoville et al., The Cosmic Evolution Survey (Cosmos): Overview, The Astrophysical J. Supplement Series, vol. 172, no. 1, 2007, p E. Bertin and S. Arnouts, SEtractor: Sotware or Source Etraction, Astronomy & Astrophysics Supplement, vol. 117, June 1996, pp B. Catanzaro et al., SEJITS: Getting Productivity and Perormance with Selective Embedded JIT Specialization, 2009; _SEJITS_Getting_Productivity_and_Perormance_With _Selective_Embedded_JIT_Specialization. 7. A. Rubinsteyn et al., Parakeet: A Just-in-Time Parallel Accelerator or Python, Proc. 4th Useni Con. Hot Topics in Parallelism, 2012; conerence/hotpar12/hotpar12-inal37.pd. 8. M. Wilde et al., Swit: A Language or Distributed Parallel Scripting, Parallel Computing, vol. 37, no. 9, 2011, pp T. Oinn et al., Taverna: Lessons in Creating a Worklow Environment or the Lie Sciences, Concurrency and Computation: Practice and Eperience, vol. 18, no. 10, 2006, pp K.D. Cooper and L. Torczon, Engineering a Compiler, Morgan Kaumann, Stean C. Müller is a PhD student in computer science in the Systems Group at ETH Zürich and a research associate at the Institute o 4D Technologies o the University o Applied Sciences Northwestern Switzerland. His research interests include big data computer systems or natural sciences and machine learning. Müller received an MSc in computer science rom ETH Zürich. Contact him at stean. mueller@hnw.ch. Gustavo Alonso is a proessor o computer science at ETH Zürich. His research interests include system sotware or modern hardware and large-scale data processing. Alonso received a PhD in computer science rom the University o Caliornia, Santa Barbara, and an engineering degree rom Madrid Polytechnic University. He is a Fellow o ACM and IEEE. Contact him at alonso@in.ethz.ch. André Csillaghy is a proessor o computer science at the University o Applied Sciences Northwestern Switzerland. His research interests include inormation integration and data analysis systems or large scientiic data collections. Csillaghy received a PhD in technical sciences rom ETH Zürich. He is a member o the IEEE Computer Society and Swiss Inormatics Society. Contact him at andre.csillaghy@ hnw.ch. Selected CS articles and columns are available or ree at SEPTEMBER