Meeting Service Level Objectives of Pig Programs∗

Zhuoyao ZhangUniversity of Pennsylvania

zhuoyao@seas.upenn.edu

Ludmila CherkasovaHewlett-Packard Labslucy.cherkasova@hp.com

Abhishek VermaUniversity of Illinois atUrbana-Champaignverma7@illinois.edu

Boon Thau LooUniversity of Pennsylvania

boonloo@cis.upenn.edu

ABSTRACTCloud computing offers a compelling platform to access alarge amount of computing and storage resources on de-mand. As the technology matures, service providers havestarted shifting their focus to support additional user re-quirements such as QoS guarantees and tailored resourceprovisioning for achieving service performance goals. An in-creasing number of MapReduce applications associated withlive business intelligence require completion time guarantees.We aim to solve the resource provisioning problem: givena Pig program with a completion time goal, estimate theamount of resources (a number of map and reduce slots) re-quired for completing the program with a given (soft) dead-line. We develop a simple yet elegant performance modelthat provides completion time estimates of a Pig program asa function of allocated resources. Then this model is used asa basis for solving the inverse resource provisioning problemfor Pig programs. We evaluate our approach using a 66-nodeHadoop cluster and a popular PigMix benchmark. The de-signed performance model accurately estimates the requiredamount of resources for Pig programs with completion timegoals: the completion times of the Pig programs with allo-cated resources are within 10% of the targeted deadlines.

1. INTRODUCTIONCloud computing has emerged as a new delivery paradigm

for providing computing services with virtually unlimitedcomputing and storage resources. The amount of enterprisedata produced daily is exploding. This is partly due to a newera of automated data generation and massive event loggingof digitized business processes. Many companies are follow-ing the new wave of using MapReduce [3] and its open-sourceimplementation Hadoop to quickly process large quantitiesof new data to drive their core business. MapReduce of-fers a scalable and fault-tolerant framework for processinglarge data sets. However, one-input data set and simple

∗This work was largely completed during Z. Zhang’s and A. Verma’s

internship at HP Labs. B. T. Loo and Z. Zhang are supported inpart by NSF grants (CNS-1117185, CNS-0845552 , IIS-0812270). A.Verma is supported in part by NSF grant CCF-0964471.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$10.00.

two-stage dataflow processing schema imposed by MapRe-duce model is a low level and rigid. To enable programmersto specify more complex queries in an easier way, severalprojects, such as Pig [5], Hive [10], Scope [2], and Dryad [7],provide high-level SQL-like abstractions on top of MapRe-duce engines. These frameworks enable complex analyticstasks (expressed as high-level declarative abstractions) to becompiled into directed acyclic graphs (DAGs) of MapReducejobs.

One of the key challenges in the cloud environment is theneed to support a performance-driven resource allocation ofcloud resources shared across multiple users running theirdata intensive programs. For instance, there is a technolog-ical shift towards using MapReduce and the above frame-works in support of latency-sensitive applications, e.g., per-sonalized advertising, sentiment analysis, spam and frauddetection, real-time event log analysis, etc. These MapRe-duce applications typically require completion time guaran-tees and are deadline-driven. Often, they are a part of anelaborate business pipeline, and they have to produce re-sults by a certain time deadline, i.e., to achieve certain per-formance goals and service level objectives (SLOs). Whilethere have been some research efforts [12, 14, 9] towardsdeveloping performance models for MapReduce jobs, thesetechniques do not apply to complex queries consisting ofMapReduce DAGs that characterize Pig programs.

In this paper, we study the popular Pig framework [5] anddesign a performance modeling environment for Pig pro-grams as a solution to the resource provisioning problem.Specifically, given a Pig program with a completion timegoal, we aim to estimate the amount of resources (a numberof map and reduce slots) required for completing the Pigprogram with a given (soft) deadline. Once this estimate isknown, the appropriate number of slots can be allocated tothe program for the duration of its execution. As a basisfor addressing this problem, we first develop a performancemodel that allows us to estimate the completion time of aPig program as a function of allocated resources.

We focus on Pig, since it is quickly becoming a popularand widely-adopted system for expressing a broad variety ofdata analysis tasks. With Pig, the data analysts can specifycomplex analytics tasks without directly writing Map andReduce functions. In June 2009, more than 40% of Hadoopproduction jobs at Yahoo! were Pig programs [5].

In addressing the outlined problem, the paper makes thefollowing contributions. We propose an intuitive and ef-ficient performance model that predicts a completion timeand required resource allocation for a Pig program with per-formance completion time goals. Our model uses MapRe-duce job profiles developed automatically from previous runs

of the Pig program. The profiles reflect critical performancecharacteristics of the underlying application during its exe-cution, and are built without requiring any modifications orinstrumentation of either the application or the underlyingHadoop/Pig execution engine. All this information can beautomatically obtained from the counters at the job mas-ter during the job’s execution or alternatively parsed fromthe job logs. Our evaluation of the proposed performancemodel is done using the popular PigMix benchmark [1] on a66-node Hadoop cluster. It shows that the predicted com-pletion times are within 10% of the measured ones, and thatthe proposed performance model results in effective and ro-bust resource allocation decisions.

This paper is organized as follows. Section 2 providesa background on Pig framework. Section 3 introduces anovel performance model for Pig programs with completiontime goals. This model is evaluated in Section 4. Section 5describes the related work. Section 6 presents a summaryand future directions.

2. BACKGROUND: PIG PROGRAMSThere are two main components in the Pig system:

• The language, called Pig Latin, that combines high-leveldeclarative style of SQL and the low-level procedural pro-gramming of MapReduce. A Pig program is similar tospecifying a query execution plan: it represents a sequenceof steps, where each one carries a single data transforma-tion using a high-level data manipulation constructs, likefilter, group, join, etc. In this way, the Pig program en-codes a set of explicit dataflows.

• The execution environment to run Pig programs. The Pigsystem takes a Pig Latin program as input, compiles itinto a DAG of MapReduce jobs, and coordinates theirexecution on a given Hadoop cluster. Pig relies on under-lying Hadoop execution engine for scalability and fault-tolerance properties.

The following specification shows a simple example of a Pigprogram. It describes a task that operates over a table URLsthat stores data with the three attributes: (url, cate-gory, pagerank). This program identifies for each cate-gory the url with the highest pagerank in that category.

URLs = load ’dataset’ as (url, category, pagerank);groups = group URLs by category;result = foreach groups generate group, max(URLs.pagerank);store result into ’myOutput’

The example Pig program is compiled into a single MapRe-duce job. Typically, Pig programs are more complex, andcan be compiled into an execution plan that consists ofseveral stages of MapReduce jobs, some of which can runconcurrently. The structure of the execution plan can berepresented by a DAG of MapReduce jobs that could con-tain both concurrent and sequential branches. For exam-ple, Figure 1 shows a possible DAG of five MapReduce jobs{j1, j2, j3, j4, j5}, where each node represents a MapReducejob, and the edges between the nodes represent the datadependencies between jobs.

j1

j2

j3

j4

j5 j6

Figure 1: Example of a Pig program’ execution planrepresented as a DAG of MapReduce jobs.

To execute the plan, the Pig engine will first submit all theready jobs (i.e., the jobs that do not have data dependencyon the other jobs) to Hadoop. After Hadoop has processedthese jobs, the Pig system will delete those jobs and the cor-responding edges from the processing DAG, and will identifyand submit the next set of ready jobs. This process contin-ues until all the jobs are completed. In this way, the Pigengine partitions the DAG into multiple stages, each con-taining one or more independent MapReduce jobs that canbe executed concurrently.

For example, the DAG shown in Figure 1 will be parti-tioned into the following four stages for processing:

• first stage: {j1, j2};• second stage: {j3, j4};• third stage: {j5};• fourth stage: {j6}.

3. PIG PERFORMANCE MODELWe begin with a performance model for estimating the

completion time of a Pig program. Then we offer the effi-cient solution for the inverse problem: finding the resourceallocation for a Pig program with performance goals.

3.1 Performance Model of a Single MapRe-duce Job

As a building block for modeling Pig programs definedas DAGs of MapReduce jobs, we apply a slightly modifiedapproach introduced in [12] for performance modeling of asingle MapReduce job. The proposed MapReduce perfor-mance model [12] evaluates lower and upper bounds on thejob completion time. It is based on a general model forcomputing performance bounds on the completion time ofa given set of n tasks that are processed by k servers, (e.g.,n map tasks are processed by k map slots in MapReduceenvironment). Let T1, T2, . . . , Tn be the duration of n tasksin a given set. Let k be the number of slots that can eachexecute one task at a time. The assignment of tasks to slotsis done using an online, greedy algorithm: assign each taskto the slot which finished its running task the earliest. Letavg and max be the average and maximum duration of then tasks respectively. Then the completion time of a greedytask assignment is proven to be at least:

T low = avg · nk

and at most

Tup = avg · (n− 1)

k+ max.

The difference between lower and upper bounds representsthe range of possible completion times due to task schedul-ing non-determinism (i.e., whether the maximum durationtask is scheduled to run last). Note, that these provablelower and upper bounds on the completion time can be eas-ily computed if we know the average and maximum dura-tions of the set of tasks and the number of allocated slots.See [12] for detailed proofs on these bounds.

As motivated by the above model, in order to approximatethe overall completion time of a MapReduce job J , we needto estimate the average and maximum task durations duringdifferent execution phases of the job, i.e., map, shuffle/sort,and reduce phases. Measurements such as MJ

avg and MJmax

(RJavg and RJ

max) of the average and maximum map (reduce)task durations for a job J can be obtained from the execu-tion logs. By applying the outlined bounds model, we can

estimate the completion times of different processing phasesof the job: map, shuffle/sort, and reduce phases. For exam-ple, let job J be partitioned into NJ

M map tasks. Then thelower and upper bounds on the duration of the entire mapstage in the future execution with SJ

M map slots (denotedas T low

M and TupM respectively) are estimated as follows:

T lowM = MJ

avg ·NJM/SJ

M (1)

TupM = MJ

avg · (NJM − 1)/SJ

M + MJmax (2)

Similarly, we can compute bounds of the execution time ofother processing phases of the job. As a result, we can ex-press the estimates for the entire job completion time (lowerbound T low

J and upper bound TupJ ) as a function of allo-

cated map/reduce slots (SJM , SJ

R) using the following equa-tion form:

T lowJ =

AlowJ

SJM

+Blow

J

SJR

+ ClowJ (3)

The equation for TupJ can be written in a similar form (see [12]

for details and exact expressions for the coefficients in theseequations). Typically, the average of lower and upper bounds(T avg

J ) is a good approximation of the job completion time.Once we have a technique for predicting the job comple-

tion time, it also can be used for solving the inverse problem:finding the appropriate number of map and reduce slots thatcould support a given job deadline D. For example, by set-ting the left side of Equation 3 to deadline D, we obtainEquation 4 with two variables SJ

M and SJR.

D =Alow

J

SJM

+Blow

J

SJR

+ ClowJ (4)

Note, that Equation 4 yields a hyperbola if SJM and SJ

R

are the variables. All integral points on this hyperbola arepossible allocations of map and reduce slots which result inmeeting the same deadline D.

0

20

40

60

80

100

0 20 40 60 80 100

Num

ber

of r

educ

e sl

ots

Number of map slots

A

B

C

0

50

100

150

200

0 20 40 60 80 100Sum

of m

ap a

nd r

educ

e sl

ots

Number of map slots in (map, reduce) Configuration

AB

C

Figure 2: Lagrange curve

As shown in Figure 2 (left), the allocation could use themaximum number of map slots and very few reduce slots(shown as point A) or very few map slots and the maximumnumber of reduce slots (shown as point B). These differentresource allocations lead to different amount of resourcesused (as a combined sum of allocated map and reduce slots)shown Figure 2 (right). There is a point where the sumof the map and reduce slots is minimized (shown as pointC). We calculate this minima on the curve using Lagrange’smultipliers [12], since we would like to conserve the numberof map and reduce slots required for the minimum resourceallocation per job J with a given deadline D. Note, thatwe can use D for finding the resource allocations from thecorresponding equations for upper and lower bounds on thejob completion time estimates. In Section 4, we will comparethe outcome of using three alternative performance metricsfor estimating a completion time of a Pig program.

3.2 Performance Model for Pig ProgramsUsing the model of a single MapReduce job as a building

block, we consider a Pig program P that is compiled into aDAG of |P | MapReduce jobs P = {J1, J2, ...J|P |}.Automated profiling. To automate the constructionof all performance models, we build an automated profil-ing tool that extracts the MapReduce job profiles1 of thePig program from the past program executions. These jobprofiles represents critical performance characteristics of theunderlying application during all the execution phases: map,shuffle/sort, and reduce phases.

For each MapReduce job Ji(1 ≤ i ≤ |P |) that constitutes

Pig program P , in addition to the number of map (NJiM ) and

reduce (NJiR ) tasks, we extract metrics that reflect durations

of map and reduce tasks (note that shuffle phase measure-ments are included in reduce task measurements) 2:

(MJiavg,M

Jimax, AvgSizeJiinput

M , SelectivityJiM )

(RJiavg, R

Jimax, Selectivity

JiR )

• AvgSizeJiinputM is the average amount of input data per

map task of job Ji (we use it to estimate the number ofmap tasks to be spawned for processing a new dataset).

• SelectivityJiM and SelectivityJi

R refer to the ratio of themap (and reduce) output size to the map input size. Itis used to estimate the amount of intermediate data pro-duced by the map (and reduce) stage of job Ji. This allowsto estimate the size of the input dataset for the next jobin the DAG.

Completion time bounds. Using the model outlined inSection 3.1, and the knowledge on the number of map andreduce slots (SJi

M , SJiR ) that were allocated for the execution

of job Ji in the Pig program P , we can approximate thelower bound of completion time of each job Ji withing Pigprogram P , as a function of (SJi

M , SJiR ) (i = 1, ..., |P |).

T lowJi

(SJiM , SJi

R ) =Alow

Ji

SJiM

+Blow

Ji

SJiR

+ ClowJi

(5)

Therefore, we can estimate the overall completion time ofthe Pig program P as a sum of completion times of all thejobs that constitute P :

T lowP =

∑1≤i≤|P |

T lowJi

(SJiM , SJi

R ) (6)

The computation of the estimates based on different bounds(Tup

P and T avgP ) are handled similarly: we use the respective

models for computing TupJ or T avg

J for each MapReduce jobJi(1 ≤ i ≤ |P |) that constitutes Pig program P .

1To differentiate MapReduce jobs in the same Pig program, we mod-

ified the Pig system to assign a unique name for each job as follows:queryName-stageID-indexID, where stageID represents the stage inthe DAG that the job belongs to, and indexID represents the indexof jobs within a particular stage.2Unlike prior models [12], we normalize all collected measurements

per record to reflect the processing cost of a single record. Thisnormalized cost is used to approximate the duration of map and re-duce tasks when the Pig program executes on a new dataset with alarger/smaller number of records. The task durations are computedby multiplying the measured per-record time by the number of inputrecords processed by the task.

3.3 Estimating Resource AllocationConsider a Pig program P = {J1, J2, ...J|P |} with a given

completion time goal D. The problem is to estimate the re-quired resource allocations (the set of map and reduce slotsallocated to P during its execution) that enable the Pig pro-gram P to be completed with a (soft) deadline D.

There are a few design choices for determining the re-quired resource allocation for a given Pig program. Thesechoices are driven by the bound-based performance modelsdesigned in Section 3.2:

• Determine the resource allocation when deadline D is tar-geted as a lower bound of the Pig program completiontime. Typically, this leads to the least amount of resourcesthat are allocated to the Pig program for finishing withindeadline T . The lower bound on the completion time cor-responds to “ideal” computation under allocated resourcesand is rarely achievable in real environments.

• Determine the resource allocation when deadline D is tar-geted as an upper bound of the Pig program completiontime. This would lead to a more aggressive resource al-locations and might result in a Pig program completiontime that is much smaller (better) than D because worstcase scenarios are also rare in production settings.

• Finally, we can determine the resource allocation whendeadline D is targeted as the average between lower andupper bounds on the Pig program completion time. Thissolution might provide a balanced resource allocation thatis closer for achieving the Pig program completion time D.

For example, when D is targeted as a lower bound of the Pigprogram completion time, one possible strategy is to pick aset of intermediate completion times Di for each job Ji from

the set P = {J1, J2, . . . , J|P |} such that Σ|P |i=1Di = D. In

other words, we need to solve the following equations basedon equation 5 for an appropriate pair(Si

M , SiR) of map and

reduce slots for each job in the DAG:

A1

SJ1M

+ B1

SJ1R

+ C1 = D1

A2

SJ2M

+ B2

SJ2R

+ C2 = D2

...

A|P |

SJ|P |M

+B|P |

SJ|P |R

+ C|P | = D|P |

(7)

where Ai = AlowJi·NJi

M , Bi = BlowJi·NJi

R and Ci = ClowJi

.However, such a solution would be difficult to implement

and manage by the scheduler. When each job in a DAGrequires a different allocation of map and reduce slots thenit is difficult to reserve and guarantee the timely availabilityof the required resources.

A simpler and more elegant solution would be to deter-mine a specially tailored resource allocation of map and re-duce slots (SP

M , SPR ) to be allocated to the entire Pig program

P , i.e., a single pair of map and reduce slots (SPM , SP

R ) that isallocated to each job Ji in P , 1 ≤ i ≤ |P | such that P wouldfinish within a given deadline D. Specifically, equation 7 can

be rewritten with the condition SJ1M = SJ2

M = · · · = SJ|P |M =

SPM and SJ1

R = SJ2R = · · · = S

J|P |R = SP

R as∑1≤i≤|P | Ai

SPM

+

∑1≤i≤|P | Bi

SPR

+∑

1≤i≤|P |

Ci = D (8)

By using the Lagrange’s multipliers method as describedin [12], we determine the minimum amount of resources (i.e.

a pair of map and reduce slots (SpM , Sp

R) that results in theminimum sum of the map and reduce slots) that needs tobe allocated to P for completing with a given deadline D.

Solution when D is targeted as an upper bound or an av-erage between lower and upper bounds of the Pig programcompletion time can be found in a similar way.

4. EVALUATION OF THE PIG MODELWe present evaluation results to validate the accuracy of

the introduced Pig model: (1) its ability to predict the com-pletion time of a Pig program as a function of allocatedresources, and (2) the accuracy of resource allocation deci-sions for meeting the specified deadlines. We describe theexperimental testbed and a workload used in our case study.

4.1 Experimental Testbed and WorkloadAll experiments are performed on 66 HP DL145 GL3 ma-

chines. Each machine has four AMD 2.39GHz cores, 8 GBRAM and two 160GB hard disks. The machines are set upin two racks and interconnected with gigabit Ethernet. Weused Hadoop 0.20.2 and Pig-0.7.0 with two machines dedi-cated as the JobTracker and the NameNode, and remaining64 machines as workers. Each worker is configured with 2map and 1 reduce slots. The file system blocksize is set to64MB. The replication level is set to 3. We disabled specu-lative execution since it did not lead to significant improve-ments in our experiments.

In our case study, we use the popular PigMix bench-mark [1] that was created for testing Pig system perfor-mance. It consists of 17 Pig programs (L1-L17), which usedatasets generated by the default Pigmix data generator.

In total, it generates 8 tables that create representativedatasets that are commonly processed by Pig programs. Pig-Mix benchmark operates over a variety of tables: from asmall table with 500 rows to a very large table with morethan 100 million records. These tables are defined by vary-ing number of fields: one table called ”widerow”contains 500fields and others contain 6-9 fields. Moreover, two diversedistributions (zipf and uniform) are used for generating thedata in these tables.

The programs in the PigMix benchmark cover a widerange of the Pig features and operators, including explodingnested data (e.g., L1), performing different join algorithms,e.g, replicate join in L2, outer join in L13, merge join in L14.These programs utilize the user defined functions (e.g., L4),or perform multi-store query (L12), or execute group-all op-erator (e.g., L8), etc. The compiled DAGs of the PigMixprograms contain both sequential branches (e.g., L3) andconcurrent branches (e.g., L11).

4.2 Completion Time PredictionFor our experiments, we create two different datasets:

• a test dataset that is generated by the PigMix benchmarkdata generator. It contains 125 million records for thelargest table and has a total size around 1TB.• an experimental dataset that is formed by the same layout

tables as the test dataset but which are 20% larger.

We run the Pigmix benchmark on the test dataset and usethese executions for extracting the job profiles of the cor-responding Pig programs. After that, using the extractedjob profiles and the designed Pig model we compute thecompletion time estimates of Pig programs in the PigMixbenchmark for processing these two datasets (test and ex-perimental) as a function of allocated resources. We validatethe predicted completion times against the measured ones.

Figure 3 shows the predicted vs measured results for thePigMix benchmark that processes the test dataset with 128map and 64 reduce slots. Given that the completion timesof different programs in PigMix are in a broad range of 100s– 2000s, for presentation purposes and easier comparison,we normalize the predicted completion times with respectto the measured ones. The three bars in Figure 3 representthe predicted completion times based on the lower (T low)and upper (Tup) bounds, and the average of them (T avg).We observe that the actual completion times (shown as thestraight Measured-CT line) of all 17 programs fall betweenthe lower and upper bound estimates. Moreover, the pre-dicted completion times based on the average of the upperand lower bounds are within 10% of the measured resultsfor most cases. The worst prediction (around 20% error) isfor the Pig query L11.

0

0.5

1

1.5

2

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17

Pro

gram

Com

plet

ion

Tim

e Tlow

Tavg

Tup

Measured-CT

Figure 3: Predicted and measured completion time forPigMix executed with test dataset and 128x64 slots.

Figure 4 shows the results for the PigMix benchmark thatprocesses the test dataset with 64 map and 64 reduce slots.Indeed, our model accurate computes the program comple-tion time estimates as a function of allocated resources: theactual completion times of all 17 programs are in betweenthe computed lower and upper bounds. The predicted com-pletion times based on the average of the upper and lowerbounds provide the best results: 10-12% of the measuredresults for most cases.

0

0.5

1

1.5

2

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17

Pro

gram

Com

plet

ion

Tim

e Tlow

Tavg

Tup

Measured-CT

Figure 4: Predicted and measured completion time forPigMix executed with test dataset and 64x64 slots.

Figure 5 shows the predicted vs measured completion timesfor the PigMix benchmark that processes the larger, exper-imental dataset with 64 map and 64 reduce slots.

0

0.5

1

1.5

2

2.5

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17

Pro

gram

Com

plet

ion

Tim

e

Tlow

Tavg

Tup

Measured-CT

Figure 5: Predicted and measured completion time forPigMix executed with experimental dataset and 64x64 slots.

As shown in Figure 5, our model and computed estimatesare quite accurate. The measured completion times of allprograms are in between the low and upper bounds. Thepredicted completion times that are based on the average ofthe upper and lower bounds provide the best results: theyare within 10% of the measured results for most cases.

In Figures 3, 4, 5 we execute PigMix benchmark threetimes and report the measured completion time that is av-eraged across 3 runs. A variance for most programs in thesethree runs is within 1%-2%, with the largest variance beingaround 6%. We have omitted the error bars in Figures 3, 4, 5because the variance is so small.

4.3 Resource Allocation EstimationOur second set of experiments aims to evaluate the so-

lution of the inverse problem: the accuracy of a resourceallocation for a Pig program with a completion time goal,often defined as a part of Service Level Objectives (SLOs).

In this set of experiments, let T denote the Pig programcompletion time when the program is processed with max-imum available cluster resources (i.e., when the entire clus-ter is used for program processing). We set D = 3 · T as acompletion time goal. Using the Lagrange multipliers’ ap-proach (described in Section 3.3) we compute the requiredresource allocation, i.e., a fraction of cluster resources, atailored number of map and reduce slots that allow the Pigprogram to be completed with deadline D. As discussed inSection 3.3 we can compute a resource allocation when D istargeted as either a lower bound, or upper bound or the aver-age of lower and upper bounds on the completion time. Fig-ure 6 shows the measured program completion times basedon these three different resource allocations. Similar to ourearlier results, for presentation purposes, we normalize theachieved completion times with respect to deadline D.

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17

Pro

gram

Com

plet

ion

Tim

e Tlow-basedTavg-basedTup-based

Deadline

Figure 6: PigMix executed with the test dataset: do wemeet deadlines?

In most cases, the resource allocation that targets D as alower bound is insufficient for meeting the targeted deadline(e.g., the L17 program misses deadline by more than 20%).However, when we compute the resource allocation basedon D as an upper bound – we are always able to meet therequired deadline, but in most cases, we over-provision re-sources, e.g., L16 and L17 finish more than 20% earlier thana given deadline.

The resource allocations based on the average betweenlower and upper bounds result in the closest completion timeto the targeted program deadlines.

5. RELATED WORKWhile performance modeling in the MapReduce frame-

work is a new topic, there are several interesting researchefforts in this direction.

Polo et al. [9] introduce an online job completion time es-timator which can be used in their new Hadoop scheduler

for adjusting the resource allocations of different jobs. How-ever, their estimator tracks the progress of the map stagealone, and use a simplistic way for predicting the job com-pletion time, while skipping the shuffle/sort phase, and haveno information or control over the reduce stage.

FLEX [14] develops a novel Hadoop scheduler by propos-ing a special slot allocation schema that aims to optimizesome given scheduling metric. FLEX relies on the speedupfunction of the job (for map and reduce stages) that definesthe job execution time as a function of allocated slots. How-ever, it is not clear how to derive this function for differentapplications and for different sizes of input datasets. Theauthors do not provide a detailed MapReduce performancemodel for jobs with targeted job deadlines.

ARIA [12] introduces a deadline-based scheduler for Hadoop.This scheduler extracts and utilizes the job profiles fromthe past executions, and provides a variety of bounds-basedmodels for predicting a job completion time as a functionof allocated resources and a solution of the inverse problem.However, these models apply to a single MapReduce job.

Tian and Chen [11] aim to predict performance of a sin-gle MapReduce program from the test runs with a smallernumber of nodes. They consider MapReduce processing ata fine granularity. For example, the map and reduce tasksare each partitioned in 4 functions. The authors use a linearregression technique to approximate the cost (duration) ofeach function. The problem of finding resource allocationsthat support given job completion goals are formulated asan optimization problem that can be solved with existingcommercial solvers.

Starfish [6] applies dynamic instrumentation to collect adetailed run-time monitoring information about job execu-tion at a fine granularity: data reading, map processing,spilling, merging, shuffling, sorting, reduce processing andwriting. This detailed job profiling information enables theauthors to analyze and predict job execution under differentconfiguration parameters, and automatically derive an op-timized configuration. One of the main challenges outlinedby the authors is a design of an efficient searching strat-egy through the high-dimensional space of parameter values.The authors offer a workflow-aware scheduler that correlatedata (block) placement with task scheduling to optimize theworkflow completion time.

Ganapathi et al. [4] use Kernel Canonical Correlation Anal-ysis to predict the performance of Hive queries. However,they do not attempt to model the actual execution of theMapReduce job. The authors discover the feature vectorsthrough statistical correlation.

CoScan [13] offers a special scheduling framework thatmerges the execution of Pig programs with common datainputs in such a way that this data is only scanned once.Authors augment Pig programs with a set of (deadline, re-ward) options to achieve. Then they formulate the scheduleas an optimization problem and offer a heuristic solution.

Morton et al. [8] propose ParaTimer: the progress estima-tor for parallel queries expressed as Pig scripts [5]. Their ap-proach relies on a simplified assumption that map (reduce)tasks of the same job have the same duration. This workis closest to ours in pursuing the completion time estimatesfor Pig programs. However, the usage of the FIFO schedulerand simplifying assumptions limit the approach applicabil-ity for progress estimation of multiple jobs running in thecluster with a different Hadoop scheduler, especially if theamount of resources allocated to a job varies over time ordiffers from the debug runs that are used for measurements.

6. CONCLUSIONWith the increasing popularity of cloud computing, many

companies are now moving towards the use of cloud infras-tructures for processing large quantities of new data to drivetheir business applications. In this paper, we design a per-formance modeling framework for Pig programs as a solu-tion to the resource provisioning problem. The proposedapproach enables automated SLO-driven resource sizing andprovisioning of complex workflows defined by the DAGs ofMapReduce jobs.

In this work, the completion time of a Pig program is ap-proximated by the sum of completion times of the jobs thatconstitute this Pig program. For a Pig program defined asa sequence of MapReduce jobs this model produces accu-rate results. However, such model might over-estimate thecompletion time of DAGs with concurrent branches (dueto possible overlap of map and reduce stage executions ofconcurrent jobs). This may lead to the over-provisioned re-source allocation. In our future work, we plan to analyzesubtleties of concurrent jobs’ execution and refine the de-signed model. Our performance models are designed for thecase without node failures. We see a natural extension forincorporating different failure scenarios and estimating theirimpact on the application performance and achievable “de-graded” SLOs. We intend to apply designed models for solv-ing a broad set of problems related to capacity planning ofMapReduce applications (defined by the DAGs of MapRe-duce jobs) and the analysis of various resource allocationtrade-offs for supporting their SLOs.

7. REFERENCES[1] Apache. PigMix Benchmark,

http://wiki.apache.org/pig/PigMix, 2010.[2] R. Chaiken and et al. Easy and Efficient Parallel Processing of

Massive Data Sets. Proc. of the VLDB Endowment, 1(2), 2008.[3] J. Dean and S. Ghemawat. MapReduce: Simplified Data

Processing on Large Clusters. Communications of the ACM,51(1), 2008.

[4] A. Ganapathi and et al. Statistics-driven workload modeling forthe cloud. In Proc. of 5th International Workshop on SelfManaging Database Systems (SMDB), 2010.

[5] A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam,C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Buildinga High-Level Dataflow System on Top of Map-Reduce: The PigExperience. Proc. of the VLDB Endowment, 2(2), 2009.

[6] H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. Cetin,and S. Babu. Starfish: A Self-tuning System for Big DataAnalytics. In Proc. of CIDR’2011.

[7] M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad:Distributed Data-Parallel Programs from Sequential BuildingBlocks. ACM SIGOPS OS Review, 41(3), 2007.

[8] K. Morton, M. Balazinska, and D. Grossman. ParaTimer: aprogress indicator for MapReduce DAGs. In Proc. ofSIGMOD. ACM, 2010.

[9] J. Polo and et al. Performance-Driven Task Co-Scheduling forMapReduce Environments. In In Proc. of NOMS’2010.

[10] A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka,S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive - aWarehousing Solution over a Map-Reduce Framework. Proc. ofVLDB, 2009.

[11] F. Tian and K. Chen. Towards Optimal Resource Provisioningfor Running MapReduce Programs in Public Clouds. In Proc.of IEEE Conference on Cloud Computing (CLOUD 2011).

[12] A. Verma, L. Cherkasova, and R. H. Campbell. ARIA:Automatic Resource Inference and Allocation for MapReduceEnvironments. Proc. of ICAC’2011.

[13] X. Wang, C. Olston, A. Sarma, and R. Burns. CoScan:Cooperative Scan Sharing in the Cloud. In Proc. of the ACMSymposium on Cloud Computing,(SOCC’2011), 2011.

[14] J. Wolf and et al. FLEX: A Slot Allocation SchedulingOptimizer for MapReduce Workloads. Proc. of the 11thACM/IFIP/USENIX Middleware Conference, 2010.