Asynchronous and Anticipatory

Filter-Stream Based Parallel Algorithm for

Frequent Itemset Mining

A. Veloso1, W. Meira Jr.1, R. Ferreira1, D. Guedes1, and S. Parthasarathy2

1 Computer Science Department, Universidade Federal de Minas Gerais, Brazil{adrianov,meira,renato,dorgival}@dcc.ufmg.br

2 Department of Computer and Information Science, The Ohio-State University, USAsrini@cis.ohio-state.edu

Abstract In this paper we propose a novel parallel algorithm for fre-quent itemset mining. The algorithm is based on the filter-stream pro-gramming model, in which the frequent itemset mining process is rep-resented as a data flow controlled by a series producer and consumercomponents (filters), and the data flow (communication) between suchfilters is made via streams. When production rate matches consuptionrate, and communication overhead between producer and consumer fil-ters is minimized, a high degree of asynchrony is achieved. Our algorithmis built on this strategy − it employs an asynchronous candidate gener-ation, and minimizes communication between filters by transfering onlythe necessary aggregated information. Another nice feature of our al-gorithm is a look forward approach which accelerates frequent itemsetdetermination. Extensive experimental evaluation comproves the parallelperformance and scalability of our algorithm.

1 Introduction

The importance of data mining and knowledge discovery is growing. Fields asdiverse as astronomy, finance, bioinformatics, cyber-security are among the manyfacing the situation where large amounts of data are collected and accumulatedat an explosive rate. Dataset size impedes easy understanding and thus therehas been an increasing clamor for mining such data efficiently. Analyzing suchdatasets without the use of some kind of data reduction/mining is getting moreand more infeasible. Compact models of data, help to quickly navigate andexploit the information hidden in such huge datasets.The problem is that mining such large and potentially dynamic datasets is a

compute-intensive task, and even the most efficient of sequential algorithms maybecome ineffective. Thus, implementation of non-trivial data mining algorithmsin high performance parallel computing environments is crucial to improvingresponse times.In this paper, given the trends toward high performance data mining tech-

niques we focus on parallel mining of frequent patterns/itemsets. Much attention

2 Veloso, Meira, Ferreira, Guedes, Parthasarathy

has gone to the development of parallel algorithms for such tasks[15,2,7,6,12,8,11].However, there are yet several challenges as yet unsolved.First, parallelizing frequent itemset mining can be complicated and communi-

cation intensive. Almost all existing algorithms require multiple synchronizationpoints.Second, achieving good workload balancing in parallel frequent itemset min-

ing is extremely difficult, since the amount of computation to be performed byeach computing unit does not depend only on the amount of data assigned to it.In fact, equal-sized blocks of data (or partitions) does not guarantee equal (norapproximately equal) workloads, since the number of frequent itemsets gener-ated from each block can be heavily skewed[6,11]. Thus, an important problemthat adversely affects workload balancing is sensitivity to data skew. What isneeded is a way to prevent computing units to become idle, even in the presenceof high data skewness.Third, system utilization is an issue that is often overlooked in such ap-

proaches. The ability to make proper use of all system resources is essential inorder to provide scalability when mining frequent itemsets in huge datasets.In this paper we present a new parallel algorithm for frequent itemset min-

ing, based on the filter-stream programming paradigm. Essentially, the min-ing process can be viewed as a coarse grain data flow controlled by a seriesof components, referred to as filters. A filter receives some data from other fil-ters, performs specific processing in this data, and feeds other filters with thetransformed/filtered data. Filters are connected via streams, where each streamdenotes a unidirectional data flow from a producer filter to a consumer filter.This new approach for parallel frequent itemset mining results in interestingcontributions, which can be summarized as follows:

– We present a parallel algorithm which communicates only the necessary ag-gregate information about itemsets. Most important, the candidate gener-ation works in an asynchronous way, yielding a very effective approach fordetermining frequent itemsets.

– Our parallel algorithm is also anticipant, in the sense that it can look aheadand predict if a candidate is likely to be frequent without the necessity of ex-amining it over all partitions first. This ability becomes more effective whenthe dataset has a skewed frequent itemset distribution, a common occurrencein real workloads. In some sense, it compensates the general negative impactof data skewness in workload balancing.

– Finally, experimental demonstrates that our algorithm utilizes the availableresources very effectively and scale very well for large datasets and variousparallel configurations.

1.1 Definitions

Definition 1. [Itemsets] For any set X , its size is the number of elements inX . Let I denote the set of n natural numbers {1, 2, ..., n}. Each x ∈ I is calledan item. A non-empty subset of I is called an itemset. An itemset of size k, X= {x1, x2, ..., xk} is called a k-itemset.

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 3

Definition 2. [Transactions] A transaction T i is an itemset, where i is anatural number called the transaction identifier or tid. A transaction datasetD = {T 1, T 2, ..., T m}, is a finite set of transactions, with size |D| = m. Thesupport of an itemset X in D is the number of transactions in D that containX , given as π(X ,D) = | {T i ∈ D |X ⊆ T i}|.

Definition 3. [Frequent Itemsets] An itemset X is frequent if σ(X ,D)≥σmin, where σmin is a user-specified minimum-support threshold, with 0 <

σmin ≤ 1. The set of all frequent itemsets is denoted as F(σmin,D).

Problem 1. [Mining Frequent Itemsets] Given σmin and a transactiondataset D, the problem of mining frequent itemsets is to find F(σmin,D).

1.2 Related Work

Several parallel algorithms for frequent itemset mining were already proposedin the literature [15,2,7,6,8,11]. The majority of the proposed algorithms followone of the three main parallelizing strategies:

1. Count Distribution: This strategy follows a data-parallel paradigm inwhich the dataset is partitioned among the processing units (while the can-didates are replicated). One drawback of this strategy is that at the end ofeach iteration all nodes must exchange local absolute supports, incurring inseveral rounds of synchronization. We alleviate this problem by presentingalgorithms [14] that need only one round of synchronization by discoveringan upper bound for the global negative border [9]. FDM [6] is another algo-rithm built on the Count Distribution strategy. It employs new pruningtechniques to reduce the number of candidates processed, and consequentlythe communication is also reduced 1. Several other algorithms [10,2] alsofollow this strategy.

2. Candidate Distribution: This strategy follows a paradigm that identifiesdisjoint partitions of candidates. Candidates are partitioned based on theirprefixes, but depending on the way that candidates are partitioned, thisstrategy may suffer from poor workload balancing. ParEclat [15] is analgorithm that follows this strategy.

3. Data Distribution: This strategy attempts to maximize the use of allnode aggregate main memory, but requires to transfer the entire dataset atthe end of each iteration, incurring in very high communication overhead.

Our parallel algorithm distributes both counts and candidates. Further, ourparallel algorithm has an excellent asynchrony, and it presents several benefitsdue to the use of a new anticipation procedure.

1 Other interesting proposal introduced by the same authors is a metric for quantifyingdata skewness. For an itemset X , let pi(X ) denote the probability that X occurs inpartition i. The entropy of X is given asH = −

∑n

ipi(X )×log(pi(X )). The skewness

of X is given as S(X ) = log(n)−H(X )log(n)

, where n is the number of partitions. A dataset’stotal data skewness is the sum of the skew of all itemsets weighted by their supports.

4 Veloso, Meira, Ferreira, Guedes, Parthasarathy

2 The Filter-Stream Programming Model

The filter-stream programming model was originally proposed for Active Disks [1],to allow the utilization of the disk resident processor in a safe and efficient way.The idea was to exploit the extra processor to improve application performancein two ways. First, alleviating the computation demand on the main processorby introducing the extra, mostly idle processor. Second, it was expected thatsuch computation would reduce the amount of data that needed to be broughtfrom the disk to the main memory. The proposed model, introduced the conceptof disklets, or filters, which are entities that perceive streams of data flowing in,and after some computation it would generate streams of data flowing out.

In the model, the filters are required to declare beforehand the memory re-quired for the computation and are limited at runtime, to the input and outputstream buffers and whatever scratch space it declared, being forbidden to al-locate memory dynamicaly. Streams are abstractions for communication whichallow fixed sized untyped data buffers to be transfered from one filter to another.

In a sense, it is very similar to the concept of UNIX pipes. The differenceis that while pipes only have one stream of data coming in and one going out,in the proposed model, arbitrary graphs with any number of input and outputstreams are possible.

Later, this concept was extended as a programming model suitable for theGrid environment [5]. A runtime system, called DataCutter (DC) was then de-veloped to suport such model. Application from different domains have beenimplemented using DC obtaining good experimental results [13,4,16,?]. Creat-ing an applications in DC consists of decomposing the target application intofilters. These filters are than scheduled for execution in the machines comprisinga Grid environment.

In DC, filters are derived from a C++ class which implements basically 3functions: Initialization, Processing and Finalization. When a filter is first in-stantiated, the Initialization code is executed. Afterwards, control is transferredto the Processing method and it should stay there until the filter terminates. Itthen cleans up after itself by calling the Finalization code.

Streams in DC represent a unidirectional pipes. A filter can either write intoor read from the stream. The units of communication are fixed size buffers,agreed upon by the two sides. Each stream has a name associated with it andthe connecting of the endpoints of a stream is done at execution time. Once itis done, buffers that are written by the sender will eventually be available forreading on the reciepient side. Delivery is garantee, and there is no duplication.

The instantiation of the filters is performed by the runtime environment.One of the most important concepts in DC is that of transparent filter copies. Atexecution time, many instances of the same filter can be created. This provides asimple way to express and implement parallelism, reduce the computation timeand to balance the time spend on the several stages of the computation. So,while the concept of the decomposition of the application into filters is relatedto task parallelism, the possibility of having multiple replicas of the same filter,

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 5

on the other hand, is related to data parallelism. DataCutter nicely integratesboth forms of parallelism into one ortogonal environment.With respect to multiple copies of the same filter, or data parallelism, the

difference between one copy and another is the portion of the entire data eachcopy has seen. If the filter needs to maintain a state, it is vital that data relatedto the same portion of the entire data to be always sent to the same copy. Inother words, the problem is that of maintaining the state across multiple copiesof the same filter. Moreover, the data buffers being sent onto the streams needto be transported from one copy of the originating filter to one specific copy ofthe reciepient filter.For most cases, the selection of the actual destination of any given message

buffer actually consider the data in the message to be untyped. However, for ap-plications that maintain some state, it is important to have some understandingof the contents of the message as to decide to which copy of the destination filterneeds to be delievered. For these cases, DC implements labelled streams whichextend the notion of the buffer to a tuple < l,m > where l is a label and m isthe message. Associated with each stream there is a label domain L and a hashfunction h which maps the label from L to h(l). The label domain defines validvalues for labels in that stream and the hash function defines a domain whichmay be associated with filter replicas. In the original model, a stream maps abuffer to one of the replicas of the receiving filter in the set of running replicas.With the labeled stream, The stream can use a mapping from that value (h(l)) tothe set of replicas to decide to which copy of the filter should the buffer < l,m >

be delivered. The semantics of the mapping, which is defined by the applicationmay even include rules to instantiate new replicas dynamically to receive bufferswith a certain label, if the available replicas are not considered appropriate forsome reason.

3 Filter-Stream Based Parallel Algorithm

In this section we present our parallel algorithm for frequent itemset mining.We start by discussing the rationale of the algorithm and then we raise someimplementation issues.For sake of filter definition, we distinguish three main tasks to determine

whether an itemset is frequent or not: (1) verify whether its subsets are frequent;if so, (2) count its frequency; and (3) check whether its counting is above support.The verification filter V receives as input (represented by the stream V〉) theitemsets found to be frequent so far and determines the itemsets that should beverified as being possibly frequent (the candidates), which are the output Vo.The counter filter C receives a candidate itemset through its input stream C〉,scans the dataset and determines the support of that candidate itemset, which issent out using the stream Co. The support filter receives the support associatedwith an itemset through the stream S〉, checks whether the counter is above thesupport threshold and notifies the proper verification filter via the stream So. Wemay express the computation involved in determining the frequent itemsets by

6 Veloso, Meira, Ferreira, Guedes, Parthasarathy

instantiating the three filters for each itemset. In this case, the stream Vo and C〉are connected, as well as the streams Co and S〉. The streams So are connected tostreams V〉 according to the itemset dependence graph2. Formally, we have twolabel domains I and T associated with itemsets and transactions, respectively.The streams So and V〉 are associated with the domain I, while the others withthe itemset T .

In the context of filter-streams, there are two dimensions where the paral-lelization of frequent itemset mining algorithms may be exploited: candidate andcount. We employ both strategies in our algorithm. The verification and counterfilters, when created with multiple instances, employ a count strategy, while thesupport filter adopts the candidate division among its instances. It is interestingto notice that no dataset is replicated if we coalesce the filters, as follows. Thisstrategy puts together filters from several levels of the dependence graph (thatis, filters associated with itemsets of various sizes), using the label concept ofour programming model. Although the mapping functions in this case may notbe simple, this approach both uses the available plataform efficiently and doesnot require any replica of the transaction dataset. Further, the granularity ofthe parallelism that may be exploited is very fine, since we may assign a singletransaction or itemset to a filter, without changing the algorithm nor even itsimplementation.

The execution of the algorithm starts with the counter filters3. Each filterhas access to its local dataset partition, and the first step is to count the 1-itemsets, by scanning its partition and builds the tidsets 4 of the 1-itemsets.At this point, a label is assigned to each counted candidate, and such label iscoherent across the filters (i.e., a candidate has the same label in all filters).The next step is to discover the frequent 1-itemsets, and so each counter filtersends a pair {candidate label, local support} to a support checker filter. For eachcandidate received, the checker filter simply sums its local supports. When thisvalue reaches the minimum-support threshold, the checker filter determines thatthe candidate is frequent, and broadcasts its label to all verifier filters.

Each verifier filter receives the candidate label from the checker filter andinterprets that the candidate associated with the label is frequent. As the labelsof frequent 1-itemsets arrive at verifier filters, it is possible to start countingthe 2-itemsets that are enumerated from those frequent 1-itemsets. In order tocontrol this process, each verifier filter maintains a prefix-tree that efficientlyreturns all candidates that must be counted. Note that, because the checkerfilter communicates via broadcast and the prefixes are lexicographically ordered,all candidates are verified and counted according to the same order across thefilters, being easy to label a candidate, since its label is simply a monotonically

2 The vertices in the dependence graph are the itemsets and the edges represent whichitemsets that are subsets of a given itemset and must be frequent so that the formermay be also frequent.

3 The 1-itemsets do not need to be verified, since they do not depend on any otheritemset.

4 The set of all transaction identifiers in which a given itemset has occurred.

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 7

increasing counter. As soon as a candidate is counted, the counter filter sendsanother pair {candidate label, local support} to the checker filter, and the processcontinues until a termination condition is reached and all frequent itemsets werefound. From this brief description we distinguish three major issues that shouldbe addressed for implementing our algorithm: performance, anticipation, andtermination condition.

Performance: Each filter produces and consumes data at a certain rate. Thebest performance occurs when the instances of filters are balanced with respectto each other and the communication overhead between the filters is minimized.That is, the data production rates of the producer filters should match the datacomsuption rates of the consumer filters. In our case, the number of counterfilters must be larger than the number of the other two filters, since counterfilters perform a more computaional intensive task than the other filters.

Anticipation: A checker filter does not need to wait for all local supports ofa given candidate to determine if it is frequent. In fact, the checker filter cananticipate this information to the verifier filters, increasing the throughput ofthe checker filter and consequently accelerating the whole process. Clearly, datadistribution has a major hole in the effectiveness of the anticipation process.

Termination Condition: The execution terminates only when all filters have nomore work to be done. However, it is difficult to detect this condition because ofthe circular dependence among filters. Fortunately, our algorithm has one prop-erty that facilitates the detection of the termination condition − the candidatesare generated in the same order across filters, so that the candidate label maywork as a logical global clock, which is synchronized when all candidate labelsare equal among all filters. At this point, there is no more work to be done.

4 Experimental Evaluation

In this section we present experimental results of our parallel algorithm. Sensi-tivity analysis on our algorithm was conducted on data distribution, data size,and degree of parallelism.The hardware configuration used for the experiments is a Linux PC cluster

consisting of 50 nodes, each of which has Pentium III 650MHz processors and786MB main memory. The nodes are interconnected via Switched Fast Eth-

ernet. Message latency is assumed to be 10µsecs and the peak point-to-pointbandwidth is 100MB/second.We used both real and synthetic datasets as inputs to the experiments. The

real dataset used is called Kosarak, and it contains (anonymized) click-streamdata of a hungarian on-line news portal (Kosarak has approximatelly 900,000transactions). The synthetic datasets, generated using the procedure describedin [3], have sizes varying from 560MB (D3.2MT16I12) to 2.2GB (D12.8MT16I12).To better understand how data skewness affects our parallel algorithm, we dis-tributed the transactions among the partitions in two different ways:

8 Veloso, Meira, Ferreira, Guedes, Parthasarathy

– Random Transaction Distribution (DR): Transactions are randomly dis-tributed among equal-sized partitions. This strategy tends to reduce dataskewness, since all partitions have an equal probability to contain a giventransaction.

– Original Transaction Distribution (DO): The dataset is simply splited intoblocked partitions, preserving its original data skewness.

We start by analysing the parallel efficiency of our algorithm. We definethe parallel efficiency as: µp,q =

Tp

q/p×Tq

, where Tp is the total execution time

when p nodes are being employed. A parallel efficiency equals to 1 means linearspeedup, and when it gets above 1 it indicates that the speedup is super-linear.Table 1 shows how the parallel efficiency varies as a function of dataset size,transaction distribution and degree of parallelism. Parallel efficiency gets muchbetter when the anticipating procedure is used and dataset is larger. Parallelefficiency continues to be high even when larger degrees of parallelism, reaching7% of improvement in the best case.

Dataset Distribution Skewness Anticipating T8 (sec) µ8,16 µ16,32

D6.4MT16I12 DO 0.511562 NO 126.38 1.07 0.88

D6.4MT16I12 DO 0.511562 YES 124.11 1.07 1.06

D6.4MT16I12 DR 0.000035 NO 94.93 1.02 0.89

D6.4MT16I12 DR 0.000035 YES 92.13 1.02 0.99

D12.8MT16I12 DO 0.491903 NO 194.74 1.05 0.98

D12.8MT16I12 DO 0.491903 YES 188.18 1.07 1.07

D12.8MT16I12 DR 0.000020 NO 168.35 1.00 0.97

D12.8MT16I12 DR 0.000020 YES 165.96 1.02 1.01

Kosarak DO 0.377798 NO 642.82 0.96 0.85

Kosarak DO 0.377798 YES 639.14 1.00 0.95

Kosarak DR 0.000102 NO 640.87 0.97 0.94

Kosarak DR 0.000102 YES 639.91 0.99 0.95

Table 1. Parallel Efficiency.

We also evaluated our algorithm by means of speedup and scaleup experi-ments. Figure 1 shows speedup and scaleup numbers obtained from the syntheticand real datasets. Again, we varied the size, transaction distribution and degreeof parallelism. For the speedup experiments with synthetic data we employeddatasets with different sizes (6,400,000 and 12,800,000 transactions), and for thetwo datasets employed we observed superlinear speedups when the anticipationprocedure is used. We also observed a superlinear speedup without the anticipa-tion, but in this case the dataset has a random transaction distribution. Further,the speedup number tends to get better for larger datasets, since there is lessvariability. Impressive numbers were also observed with real data, and the bestresult was achieved with original (skewed) transaction distribution and using theanticipation procedure.

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 9

For the scaleup experiments, we varied dataset size and degree of parallelismin the same proportion. Dataset size ranges from 3,200,000 transactions (with 8filters) to 12,800,000 transactions (with 32 filters). As we can see in Figure 1, ourparallel algorithm also presents ultra scalability when the anticipation procedureis employed, or a random transaction distribution is used. Even when usingoriginal transaction distribution without anticipation, our algorithm shows tobe very scalable, reaching approximatelly 95% of scalability.

5

10

15

20

25

30

35

40

5 10 15 20 25 30 35

Va

lue

#Filters

Speedup - D6.4MT16I12 - minsupp=0.1%

idealDo - No anticipating

Do - AnticipatingDr - No anticipating

Dr - Anticipating

5

10

15

20

25

30

35

40

5 10 15 20 25 30 35

Va

lue

#Filters

Speedup - D12.8MT16I12 - minsupp=0.1%

idealDo - No anticipating

Do - AnticipatingDr - No anticipating

Dr - Anticipating

5

10

15

20

25

30

35

5 10 15 20 25 30 35

Va

lue

#Filters

Speedup - Korasak - minsupp=0.05%

IdealDo - No anticipating

Do - AnticipatingDr - No anticipating

Dr - Anticipating

0.9

0.95

1

1.05

1.1

5 10 15 20 25 30 35

Va

lue

#Filters

Scaleup - D3.2M-D12.8MT16I12 - minsupp=0.1%

IdealDo - No anticipating

Do - AnticipatingDr - No anticipating

Dr - Anticipating

Figure 1. Speedup and Scaleup Numbers (in relation to 8 nodes).

As seem in both speedup and scaleup experimets, the anticipation is moreeffective in the presence of data skewness, since the probability of a frequentitemset be sent to the verifier filter using less partitions is higher. Figure 2 showsthe average number of partitions necessary to trigger an itemset as frequent. Notethat, without any anticipation, all partitions must be analyzed, but when theanticipation procedure is used, the number of partitions that must be analyzed totrigger a frequent itemset is smaller and varies with data skewness. As expected,the same trend is also observed in the average time to trigger a frequent itemset.

10 Veloso, Meira, Ferreira, Guedes, Parthasarathy

Further, smaller itemsets usually are detected to be frequent earlier. In Table 2we present the anticipation gain, that is, how long in advance the itemset is foundto be frequent. In a 32-processor configuration using D12.8MT16I12 the gainvaried from more than 60% to 7%. This observation is particularly interestingbecause the number of short-sized itemsets is much greater than large-sizeditemsets, in particular 2 and 3-itemsets.

Itemset size 1 2 3 4 5 6

Anticipation gain 60.6% 47.0% 44.5% 16.2% 6.0% 7.2%

Table 2. Anticipation gain for D12.8MT16I12 using 32 processors.

5

10

15

20

25

30

35

5 10 15 20 25 30 35

Va

lue

#Filters

Average #Partitions - D6.4MT16I12 - minsupp=0.001%

Do - No anticipatingDo - Anticipating

Dr - No anticipatingDr - Anticipating

0

1

2

3

4

5

6

7

5 10 15 20 25 30 35

Va

lue

(se

cs)

#Filters

Average time to trigger - D6.4MT16I12 - minsupp=0.001%

Do - No anticipatingDo - Anticipating

Dr - No anticipatingDr - Anticipating

Figure 2. Effects of Anticipating: Necessary partitions and average time to trigger

In order to better understand the dynamics of the algorithm, we introducesome metrics that quantify the various phases of the algorithm. We may dividethe determination of the support of an itemset into four phases:

Activation: The various notifications necessary for an itemset become a candi-date may not arrive at the same time, and the verification filter has to waituntil the conditions for an itemset be considered candidate are satisfied.

Contention: After the itemset is considered a good candidate, it may wait inthe processing queue of the counter filter.

Counting: The counter filters may not start simultaneously, and the countingphase is characterized by counter filters calculating the support of a candi-date itemset in each partition.

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 11

Checking: The support counters of each counter filter may not arrive at thesame time in the support checker filter, and the checking phase is the timeperiod during which the notifications arrive.

Next we are going to analyze the duration of these phases in both speedupand scaleup experiments. The analysis of the speedup experiments explains theefficiency achieved, while the analysis of scaleup experiments shows the scalabil-ity.In Table 3 we show the duration of the phases we just described for con-

figurations employing 8, 16, and 32 processors for mining D12.8MT16I12. Therightmost column also shows the average processing cost for counting an itemset,where we can see that this cost reduces as the number of processors increase, asexpected. The same may be observed for all phases, except for the Activationphase, whose duration seems to reach a limit around 1 second. The problem inthis case is that the number of processors involved is high and the asynchronousnature of the algorithm makes the reduction of the Activation time very difficult.

Proc Activation Contention Counting Checking Processing

8 2.741046 5.564751 9.412093 8.469050 0.001645

16 1.264842 2.058052 4.893773 4.691232 0.000759

32 1.229330 0.273229 1.129718 1.986129 0.000369

Table 3. Speedup Experiment: Profiling (secs)

Verifying the timings for the scaleup experiments in Table 4, we verify thescalability of our algorithm. We can see that an increase in the number of proces-sors and in the size of the dataset does not affect significantly the measurements,that is, the algorithm implementation does not saturate system resources (mainlycommunication) when scaled.

Proc Activation Contention Counting Checking Processing

8 2.741046 5.564751 9.412093 8.469050 0.001645

16 2.628118 5.538353 9.349371 8.403360 0.001596

32 2.439369 5.021002 10.311501 8.906631 0.001594

Table 4. Scaleup Experiment: Profiling (secs)

5 Conclusion and Future Work

In this paper we proposed an algorithm to conduct parallel frequent itemset min-ing. The proposed algorithm is based on the filter-stream programming model,

12 Veloso, Meira, Ferreira, Guedes, Parthasarathy

where the computation of frequent itemsets is expressed as a circular data flowbetween distinct components or filters. Parallel performance is optimized, anda high degree of asynchrony is achieved by using the right number of each fil-ter. Further, we propose a very simple anticipation approach, which acceleratesfrequent itemset determination (specially in the persence of data skew). It wasempirically showed that our algorithm achieves excellent parallel efficiency andscalability, even in the presence of high data skewness.Future work includes the development of parallel filter-stream based parallel

algorithms for other data mining techniques, such as maximal/closed frequentitemsets and frequent sequential patterns. Utilization of our algorithms in realapplications is also a possible target.

References

1. Anurag Acharya, Mustafa Uysal, and Joel Satlz. Active disks: Programming model,algorithms and evaluation. In Internation Conference on Architectural Supportfor programming Languages and Operating Systems (ASPLOS VIII), pages 81–91.ACM Press, Oct 1998.

2. R. Agrawal and J. Shafer. Parallel mining of association rules. Transactions onKnowledge and Data Engineering, 8(6):962–969, 1996.

3. R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In Proc.of the Int. Conf. on Very Large Databases, VLDB, pages 487–499, SanTiago, Chile,June 1994. VLDB.

4. M. Beynon, C. Chang, U. atalyrek, T. Kur, A. Sussman, H. Andrade, R. Fer-reira, and J. Saltz. Processing large-scale multi-dimensional data in parallel anddistributed environments. Parallel Computing, 28(5):827–859, 2002.

5. Michael Beynon, Tahsin Kurc, Alan Sussman, and Joel Saltz. Design of a frame-work for data-intensive wide-area applications. In Heterogeneous Computing Work-shop (HCW), pages 116–130. IEEE Computer Society Press, May 2000.

6. D. Cheung and Y. Xiao. Effect of data distribution in parallel mining of associa-tions. Data Mining and Knowledge Discovery, 3(3):291–314, 1999.

7. E. Han, G. Karypis, and V. Kumar. Scalable parallel data mining for associationrules. Transactions on Knowledge and Data Engineering, 12(3):728–737, 2000.

8. M. Joshi, E. Han, G. Karypis, and V. Kumar. Efficient parallel algorithms formining associations. Parallel and Distributed Systems, 1759:418–429, 2000.

9. H. Mannila and H. Toivonen. Levelwise search and borders of theories in knowledgediscovery. Data Mining and Knowledge Discovery, 1(3):241–258, 1997.

10. S. Orlando, P. Palmerini, R. Perego, and F. Silvestri. An efficient parallel anddistributed algorithm for counting frequent sets. pages 421–435, 2002.

11. M. Otey, A. Veloso, C. Wang, S. Parthasarathy, and W. Meira. Mining frequentitemsets in distributed and dynamic databases. In Proc. Int. Conf. on Data Mining,ICDM, Melbourne, USA, November 2003. IEEE.

12. S. Parthasarathy, M. Zaki, M. Ogihara, and W. Li. Parallel data mining for as-sociation rules on shared-memory systems. Knowledge and Information Systems,3(1):1–29, 2001.

13. Matthew Spencer, Renato Ferreira, Michael Beynon, Tahsin Kurc, UmitCatalyurek, Alan Sussman, and Joel Saltz. Executing multiple pipelined dataanalysis operations in the grid. In Proceedings of the 2002 ACM/IEEE conferenceon Supercomputing, pages 1–18. IEEE Computer Society Press, 2002.

Asynchronous And Anticipatory Parallel Frequent Itemset Mining 13

14. A. Veloso, M. Otey, S. Parthasarathy, and W. Meira. Parallel and distributedfrequent itemset mining on dynamic datasets. In Proc. of the High PerformanceComputing Conference, HiPC, Hyderabad, India, December 2003. Springer andACM-SIGARCH.

15. M. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. New parallel algorithms for fastdiscovery of association rules. Data Mining and Knowledge Discovery, 4(1):343–373, December 1997.

16. mit atalyrek, Mike Gray, Tahsin Kur, Joel Saltz, and Renato Ferreira. Acomponent-based implementation of multiple sequence alignment. In Proceedingsof the 2003 ACM Symposium on Applied Computing, pages 122–126. ACM, 2003.