Supporting Realistic OpenMP Applications on a Commodity Cluster of Workstations

10

Transcript of Supporting Realistic OpenMP Applications on a Commodity Cluster of Workstations

�������������� ��������������������� �!�" �# $ %&���������'�(����� �� �� )�*+��,-,��/.0�1324*5����������6�87:9 ����;/����(����� ��

<>=�?A@CBEDGFIH�J0HK@�LNM�OQPR@TSUFRVW?AXYFQZKZ[HK\]LA^'?N_`PQZKa�bcHKBR=�@AXYFQ@A@�d

eQfWgGhihkjCh�lnm�jpoqfsrutwvpf1x�j]x�yGzY{hk|~}R�Rrwostcm�yG��vpyGoqostwvpyR�� �RtWzR�RoU��yRvp��ostw�uv�r��i�i�Toq��r��3x�l�xq�iosrurwok�R�����i�������1�W�1���k�girurw}N� �k�1�����'� osfqok� }G�QtWzR�Go�� oqzR�`� � x�tWxk ¡hk�Gyir

¢�£¥¤q¦q§�¨>©�¦kª �«y�rwgGvp�¬}>x�}`ost1�G�o'}Rtwoq�uosyIr¬rwoqfWgGyRv®­I�Roq��l�h�t�rutWx�yG�uj®x�rwvpyR��x�y>z�hk}R�rwvp|~vp¯qvpyG�Utwo1x�jpv®��rwvpf�°�}`oqy`  � x�}G}Rjpv®fqx�rwvphkyR�hky±zRvp��rutwvp²G�Rrwoqz(���Q��rwoq|~�q��³�gRo¬�kh�x�jh�l�h��Rt´}Rtwh�µ�oqfsr(vp�´rwh¶­i�>x�yirwv�l[��rwgGo8zRos��twoqo�rwhY�¬gGvpf·g¸°�}`osy`  � f1x�y0²`o8o·¹Q�rwoqyGzRo1z�rwh~zRvp��rutwvp²G�Qrwo1z¡���Q��rwoq|~�¬x�y>z�rwh~zQoq��oqjph�}E�u�G}R}`h�turwvpyG�±fshk|~}Gvpjpost�rwoqfWgR�yRvK­i�Roq�q�¬°��Qt�}Rtwoq�uoqyir�fqhk|~}Rvpj®o·t8rwosf·gRyGv®­i�Goq�~rutWx�yR�uj®x�rwo�°�}`oqy`  � }Rtwh���tWx�|~�vpyirwh8x(l�h�tw|º�u�Gv�rWx�²Gjpo"l[h�t�os¹Qoqfq�Rrwvph�yYhky¶x�eQh�l[r���x�two´»�eG ¼���Q��rwoq|E�R�To�g>x1��ovp|~}Gjpoq|~osyIrwoqz!x�fqhk|~}Rvpj®o·t�rwgGx�r�}`ostul�h�tw|~�8rwgRv®�8²>x��uvpf¶rutWx�yG�uj®x�rwvphkyC�¬x�yGz!��ogGx��koc}Qtwhk}`h��uo1z~hk}Rrwvp|~vp¯1x�rwv®h�y±rwoqfWgGyGv®­i�Gos�nrwgGx�r½v®|~}Qtwh���o�rwgGo¬²>x��uoqjpvpyGoc}`ostul�h�tu�|�x�yGfqo~h�l�°�}`oqy`  � x�}G}Rj®vpf1x�rwvphkyG�"hky0zQvp��rutwv®²R�Rrwo1z�fqhk|~}R�Rrwost����Q��rwoq|~�q�]°��Rttwoq�u�Rj�rw���ugGh1�¾rwg>x�r1�G�¬gGvpjpo�¿�ostwyRoqj3²`oqyRf·gR|�x�tw¿I��f1x�yY�ugGh1�ÀgRvp�kgYosÁ~fsv®osyGfs��l�h�t°�}`oqy>  � }Rtwh���tWx�|~�½h�y�zRvp��rutwvp²G�Rrwoqz±���R��rwos|~�q�kl��Rjpj>x�}R}Gjpvpf1x�rwvph�yG��yRoqo1z±f1x�twosl��Rjfqh�yG�uv®zRo·tWx�rwvphky~h�l`�ug>x�two1z±zGx�rWx�x�fqfqoq�u�n}>x�rurwostwyG�q��Â�yGx�vp��o�rutWx�yG�uj®x�rwvphky¡ÃÄ�uv®|~vpj®x�trwh±rwgRo'²>x��uvpf�rutWx�yR�ujKx�rwvphky¶zRh�yGo'²i�Y°�}`osy`  � fshk|~}Gvpjpostw�¬l�h�t�eG  � �WÅ�j®oqxkzR�¬rwhx�fqfqoq}QrWx�²Gjpo´}`ostul�h�tw|�x�yGfso´vpyY��ostu��l�os�Æx�}G}Rjpv®fqx�rwvphkyR�q�`��o"}Rtwhk}`h��uo´h�}Rrwvp|~vp¯1x��rwvphkyR���u�Rf·gEx��±ÇwÈ�É�Ê>ËIÌÎÍ�ÌÄÏÐÈ�Ñ�ÒwÓ«ÊQÍ�ÒWÌÄÏ�ÌÄÏÐÈ�ÑGÏ�ÑiÔ��>ÊQÍ1ÔiÓsÕ«Í�Ö½Í�ÒwÓ~È·Ê>ÌÄÏ[É(Ï®×1Í�ÌÄÏÐÈ�ÑGØw�Ax�yGzÍ�ÇwÇWÓ·ØwØ¥Ê>ÒWÏ�Ù�Í�ÌÄÏ®×1Í�ÌÄÏÐÈ�Ñ�rwg>x�r]twoq�u�Rjpr�vpy´x1��ostWx���o¬���kÚ�}`ostul[h�tw|�x�yRfqovp|~}Qtwh��koq|~oqyirh�y¡rwgGo�e � m½{Û°'  �  Y���k�I�´²`osyGfWgG|�x�tw¿�x�}G}Rj®vpf1x�rwvphkyG�q�

Ü Ó·Ý�Ö½È�ÒuÞ�Øn��°�}`oqy`  � Â�}R}Gjpvpf1x�rwvph�yG�q��eQh�l[r«��x�two½»Uvp��rutwvp²G�Rrwoqz´eIg>x�two1z� ¡oq|~h�tu�i�²`oqyRf·gR|�x�tw¿I�q�Q}`ostul�h�tw|�x�yRfqo'f·gGx�tWx�fsrwostwvp��rwvpfq�q�>h�}Rrwvp|~vp¯1x�rwvphkyR�

ß àGácâ`ãAä'å´æ"ç3âAèwäcá

é´êN=�@CJ0ëíì®î�ï¥ðCFQVU=kVwñqFIòAZ[H[V·ðA=k_/HKñsV·=�ZKa�FRV�FQ@�H[XEêNPRóWñqFI@Gñ'XE=�ñ·ðCP>_TFI@N_¶Z�FI@ABR?CFIBR="=�ôGñs=�@`õVWH[PQ@!aÐPQó8êAó·PRBQóqFIXEXEHK@CB�V·ðCFQó·=k_>õ�XE=�X¡PRó·O�êNFIóqFIZ[ZK=�Z¬ö�PQXEêA?Añ·=�óqV�÷�øÆðAH[ZK=Té´ê3=�@NJ0ëùðCFRVö�Z[=�FIó�FQ_`úiFI@GñqFIBQ=kVPQ@/V·ðCFIós=�_>õ�XE=�XEPQósO~êAZ�Fiñ·aÐPQósXEV�LIXE=�VsVsFIBQ=UêCFRV·V·H[@AB±H[V¬ñ·P`_AF�O�VWñ·H[Z[ZAñ·ðC=X¡PGVwñ�û'H[_`=�ZKOGõ�?CV·=�_¡êAósPQBQóqFIXEXEH[@AB´êNFIóqFQ_`H[BQXºaÐPQóc_`H�VWñ·ósHKòA?Añ·=�_`õ�XE=�XEPRó·O¡ö�PQXEêA?Añ·=�óqV�÷Iüu@ñ·ðAH�V�êCFQêN=�ó�Liû�='=�ô`êAZ[PQós=cñsðA=�V·?AH®ñqFIòAH[Z[H®ñwO�PIa¥é´ê3=�@NJ0ë!aÐPQó¬_`H�VWñ·ósHKòA?Añ·=�_EV·O`Vwñs=�XYV�FQV�û�=�Z[Z«÷

ý ³�gGvp��|�x�rwostwv®x�jNv®�c²>x��uo1zE�G}`h�yE��h�tw¿¡�u�G}R}`h�turwo1z�v®yY}Gx�tur�²i��rwgRo'�Ux�rwvph�y>x�j¥eQfqvpoqyRfqo"þGhk�RyR�zGx�rwv®h�yÿ�GyGzRost���tWx�yir���hR��������I�1�k�I�c�k�k�k�k���k�i�¬�k�����k�k���I�cx�y>z��k�k�����k���I�cÂ�yi�0hk}Rv®yRvphkyG�q��>yGzRvpyG���q�3x�y>z¶fqhkyRfqjp�G�uvphkyR�"h�t�twoqfqhk|~|~osy>zGx�rwvphkyG��os¹Q}Rtwoq�u�uo1z¶vpy�rwgGvp�'|�x�rwostwv®x�j½x�two±rwgGh��uoh�lCrwgGocx��QrwgGh�tw��x�yGz�zRh�yGh�ryRoqfqoq�u�wx�twv®j��~two��Goqfsr½rwgGoc�Qvpos�¬�½h�lCrwgGo¬�Ux�rwvphky>x�jCeQfqvpoqyRfqo�þGhk�RyR�zGx�rwv®h�yN�

é´?Aó�òCFQV·H�ö'FIêAêAósPRFRöqð8H�V�ñ·P~?NVW=�F�<>PQaÄñwû�FQó·=��±<`J����´H�Vwñsó·H[òA?`ñs=�_/<>ðCFQó·=k_�J0=�XEPQósO���VWO`VWõñ·=�X�LAû'ðCH[öqðTêAósPiúGH�_`=kVcñsðA=~ú>HK=�û PQaFEVWðNFIós=�_¶FR_A_`ós=�VsVUVWêCFRö�=±PQ@�ñ·PRê�PQaFE_`H�Vwñsó·H[òA?`ñs=�_>õX¡=�XEPQósO�FIóqöqðAHKñ·=kö1ñ·?Có·=R÷��nPEñsðAH[V"=�@C_�LCû�=~ðCF�úQ=~H[XEêAZK=�XE=�@Gñ·=k_�FYö�PQXEêAH[Z[=�ó'ñsðCFiñ�ñsósFQ@CVWõaÐPQósXEV¶é´ê3=�@NJ0ë+êAó·PRBQóqFIXYV�HK@Gñ·P��½ó·=kFQ_`XYFQó·\`VE<>PIaÄñwûUFIós=�±<`J êAósPQBRósFQXEVTì �ï�÷��UðAH[VêCFIê3=�ó'XYFI\R=�V�ñsðA=±aÐPQZ[ZKPiû'H[@ABEVWê3=�ö�H �3ö~ö�PQ@Gñsó·H[òA?`ñsHKPR@CV�÷

� ø�=E_`=kV·ö�ó·H[òN=�PQ?Aó(ö�PRX¡êCHKZ[=�ó(HK@AaÐósFRVwñsó·?Cö�ñ·?Aós=~ñsðCFiñ~ö�FQ@�ñ·óqFI@CV·Z�Fiñ·=Eé´êN=�@CJ0ë FQêAêAZ[H®õö�FIñ·H[PQ@CVUH[@Gñ·P/<>PIaÄñwûUFIós=��(<AJ êCó·PRBQóqFIXYV�÷� ø�= XE=�FRVW?Có·=¼ñsðA= òCFQV·=�Z[H[@A= ê3=�ó·aÐPQósXYFI@Cö�= PIa ñ·ðA= FIêAêCZKH�ö�FIñ·H[PQ@`õ�ZK=�úQ=�Z <`ë�b��é(J0ëcJ�����Cî�ò3=�@CöqðAXYFQó·\`V´PQ@�F�_`H�Vwñsó·H[òA?`ñs=�_�X¡=�XEPQósO�V·O`Vwñs=�X�L¥FI@C_¸û�=EFI@NFIZ[O���=ñ·ðA=~ê3=�ó·aÐPQósXYFI@Cö�=±òN=�ðCF�ú>HKPRó�÷

� ø�=�êAós=�V·=�@Gñ"PQêAñ·H[X¡H���FIñ·H[PQ@�ñs=�öqðA@AH��G?A=�V�ñ·ðCFIñ´=�@AðCFI@Nö�=~ñsðA=8òCFRVW=�ZKH[@A=�ê3=�ó·aÐPQósXYFI@Cö�=PIa�ó·=kFIZné´ê3=�@CJ0ëÆFQêAêAZ[H[ö�FiñsHKPR@CV�PQ@�_`H�Vwñsó·H[òA?`ñs=�_TXE=�XEPQósO¶V·O>VWñ·=�X�÷é´?Aó�û�PQós\/H[V´ö�Z[PRV·=�Z[O¶ós=�Z�Fiñs=�_�ñ·PYñsðA=~aÐPRZKZ[Piû'HK@CBYêAó·P��w=�ö�ñsV�÷Aüu@Ûì �Iï�LCñ·ðA=�FI?`ñsðAPQóqV'H[X�õ

êAZK=�XE=�@Gñ·=k_Eé´ê3=�@CJ¸ë¸PR@�F´@A=�ñwû�PQós\~PIaNV·ðCFQó·=k_�XE=�XEPQósO±X8?AZKñ·H[êAósP`ö�=�VsV·PQóqV½FQ@C_8V·ðAPiû�=�_ñ·ðA=�HKó'ê3=�ó·aÐPQósXYFI@Cö�=´?CV·H[@ABYF�V·?AòCV·=�ñ'PIa½ñ·ðA=8<Aë��nM(< �'õ!�FI@C_""M(<¶ò3=�@NöqðAXYFIós\/VW?AHKñ·=kV�Lû'H®ñsðAPQ?`ñ~ñsðA=YðA=�ZKêÿPIa'ö�PRXEêAHKZ[=�ó~PRê`ñ·H[XEH#�kFiñsHKPR@�÷�üu@ ì $Qï�Lnö�PQXEêAH[ZK=�ó~PQê`ñsHKXEH���FIñ·H[PQ@CV±ðCF�úR=òN=�=�@YH[@Rñsó·P`_`?Nö�=�_�aÐPRó�é´êN=�@CJ0ë êCó·PRBQóqFIXYVPQ@/<>PIaÄñwûUFIós=��(<AJ ÷><`=�úQ=�ósFQZ`ó·=kö�=�@Rñ¬êNFIê3=�óqVðCF�úQ=~êAósPQê3PRV·=�_TZ�FI@ABR?CFIBR=~=�ô>ñ·=�@CV·HKPR@CV�÷&%APQó´=�ôAFIXEêAZ[=QLCHK@¾ì ')(+*kï�L`ñ·ðC=8FI?`ñsðAPQóqV�_`=kV·ö�ó·H[ò3=_AFiñqF�_`H[VWñ·ósH[òA?`ñ·H[PQ@ _`H[ós=�ö1ñsHKúR=�V~V·HKXEH[Z[FQó±ñ·P�ñsðA=/PQ@C=�V~_`=kVWH[BQ@A=k_�aÐPQó,�"HKBRð`õ�ë�=�óWaÐPRó·XYFQ@Cö�=%APQó·ñ·óqFI@-���´ë�%.�~ì /iï�÷

%AósPQX ñ·ðC=�V·=Tós=�Z�Fiñs=�_�=10]PQó·ñsV�L�û�=�aÐPQ?C@C_�ñ·ðCFIñ�L�û'ðAH[ZK=�ós=�V·?AZKñsV�aÐósPQX V·XYFIZ[Z'\Q=�ós@A=�ZêAó·PRBQóqFIXYV�ðCF�úR="VWðCPiû'@YêAó·PRXEH[V·HK@CB~êN=�óWaÐPRó·XYFI@Nö�=QLRZKHKñWñsZK=´HK@AaÐPQósXEFIñ·H[PQ@YPR@Eñ·ðA=´òN=�ðCF�ú>HKPRóPIa½ós=�FQZKH�VWñ·H�ö~é´ê3=�@NJ0ë FIêAêAZ[H�ö�FiñsHKPR@CVUPQ@�<>PQaÄñwû�FQó·=2�(<AJ V·O`Vwñs=�XYVUH�V'F�ú�FQHKZ�FIòCZK=R÷Güu@TñsðAH[VêCFIê3=�ókLnû�=¶V·ðAPiû ðCPiû5FQêAêAZ[H[ö�FiñsHKPR@`õ�Z[=�úR=�Z¬ò3=�@NöqðAXYFIós\>V�êN=�óWaÐPRó·XYV~PR@¾<>PIaÄñwûUFIós=3�(<AJFI@C_ÿêAósPQê3PRV·=EPQê`ñsHKXEH���FIñ·H[PQ@!ñ·=köqðA@AH��R?C=�V~ñsP�HKXEêAósPiúQ=EñsðA=¶V·êN=�=�_`?CêCV�÷M�Z�VWPNL¥ñ·ðC=/PQê`ñsH®õX¡H���FIñ·H[PQ@¶ñ·=�öqðC@AH��G?A=kV�ñ·ðCFIñ�û�=(FQó·=´êAós=�V·=�@Gñ·H[@AB8H[@/ñsðAH�VUêCFIê3=�ó�?CV·=(VWñsFQ@C_AFIóq_�é´ê3=�@CJ¸ëFQVUH[@AêA?`ñ"FQ@C_�_`PE@APIñ�ós=�Z[O/PQ@�ñ·ðC=~?CVW=�ó54 V'_AFIñsFE_`H�Vwñsó·H[òA?`ñsHKPR@TH[@AêA?Añ�÷

�UðA="êCFQêN=�ócH�V�PQósBRFI@CH#��=�_EFQV¬aÐPQZ[ZKPiû�V�÷><`=�ö1ñsHKPR@6±û'H[ZKZ]êAós=�V·=�@GñcòNFQV·H[ö´ö�PQXEêAH[Z[=�ó¬ñs=�öqð`õ@AH��G?A=kV/aÐPRó/ñsósFQ@CVWZ�FiñsHK@CBÀé´êN=�@CJ0ë HK@Gñ·PÀ<>PIaÄñwûUFIós=7�±<`J êAósPQBRósFQXYV�÷c<>=kö1ñsHKPR@8�Ûû'H[ZKZ_`H[Vsö�?NV·V(ñsðA=YêN=�óWaÐPRó·XYFQ@Cö�=Eò3=�ðCF�ú>H[PQó±PIaUVW?CöqðÿêAósPQBRósFQXYV�÷¥<>=�ö�ñ·H[PQ@9$�û'HKZ[Z¬êAós=�V·=�@Gñ�FQ_`õú�FQ@Cö�=k_EPQê`ñsHKXEH���FIñ·H[PQ@CV�÷Rüu@¶<`=�ö1ñsHKPR@6'`LRû�="û'HKZ[Z:�G?CFQ@Gñ·HKñsFiñsHKúR=�Z[O�=�úiFIZ[?CFiñs=�PQ?Aó�êAósPQê3PRV·=�_ñ·=�öqðC@AH��G?A=kV�L>aÐPRZKZ[Piû�=�_�òGO�ö�PQ@Nö�Z[?CVWH[PQ@NVUHK@0<>=kö1ñ·H[PQ@<;A÷

= >¡ã�?�áA@CBD?½âAè·áFEHG�I�J½áAK�LNMOIPIPB·èWçQ?âAèwäcáP@ÛèWá�â`äSRUä�Tsâ�U9?ã+JWVWRXKL�ãAä�E¬ã�?.YZ@

üu@ÿñ·ðA=YñsósFQ@CV·H®ñsHKPR@�aÐósPQX VWðNFIós=�_>õ�XE=�XEPQósO�ñ·P¸_AH[VWñ·ósHKòC?`ñ·=k_ VWO`VWñ·=�XYV�L¥ñ·ðC=/XYF[�wPQó�öqðNFIZKõZK=�@ABQ=�H�VEñ·ðCFIñ/=�FRöqð�êCFQóWñsH[ö�HKêNFiñ·H[@ABÿ@AP`_`=�@APiû ðCFQVEHKñsV/Piû'@ÀêCó·H[úiFiñ·=�FR_A_`ós=�VsVYVWêCFRö�=RLû'ðAH[öqð�H�VU@APIñ�ú>H�VWH[òAZ[=(ñ·PYPIñsðA=�ó'@CP>_A=�V�÷+�UðAH�V'_`H 0]=�ós=�@Nö�=~H[@�FR_A_`ós=�VsVUVWêCFRö�=kV'F[0]=�ö1ñqVUñ·ðC=é´êN=�@CJ0ë¾ñ·óqFI@CV·Z[FIñ·H[PQ@�÷>üu@Tö�FQV·=(PIa�<AJ0ëcV�L>HKXEêAZ[=�XE=�@GñsHK@AB¶é´ê3=�@CJ¸ë�V·ðCFIós=�_/ú�FQó·H�FIòCZK=kVH[VVwñsósFQHKBRðGñWaÐPQósûUFIóq_¥L1òN=kö�FQ?CVW=�FIZ[ZRúiFIósH�FIòAZ[=�V�FQó·=�FQö�ö�=�VsVWH[òAZ[=¬ò>O(FQZKZRñ·ðCó·=kFQ_AV�÷�S�O±ö�PQ@GñsósFRVwñkLHK@�<`PIaÄñwûUFIós=2�±<`J V·O`Vwñs=�XYV�LNFIZ[ZnúiFQó·H�FIòAZ[=�V"FIós=~êAó·H[úiFiñs=�ò>OT_`=�aÎFI?AZKñ�L]FI@N_�VWðCFQó·=k_�_AFiñqFðCFQVñsP~òN='=�ô>êCZKH�ö�HKñ·Z[O�FQZKZ[P`ö�Fiñs=�_¡FQV¬V·?Cöqð�÷��UðCH[V¬ö�ó·=kFiñs=�V�F±öqðCFIZ[Z[=�@ABR=�aÐPRó�ñ·ðA=UñsósFQ@CV·Z[FIñ·H[PQ@PIa�XEPGVwñ8é´êN=�@CJ0ë êAósPQBQóqFIXYV�LNû'ðC=�ós=�úiFIósH�FIòAZ[=�V(FIós=¡V·ðCFQó·=k_0ò>O0_`=�aÎFQ?AZKñ�÷\�½PTFR_A_`ós=�VsV

SPEC OMP2001M Baseline Performance

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

wupwise swim mgrid applu equake art

Spee

dup

1 Processor 2 Processor 4 Processor 8 Processor

����� ª��>ª�� x��uosj®vpyRo�}`ostul[h�tw|�x�yGfqo�h�l`l�h��Rt½þGh�turutWx�y��k��x�y>z"r���h'{�²`osyGfWgG|�x�tw¿I�3lKtwhk|ÆrwgGo�e � m{°'  �  Y�����Q�"²`oqyRf·gR|�x�tw¿~�u�Gv�rwo'h�y��k�`�I�R�R�Gx�yGzE�(|�x�fWgGvpyRoq�q�

ñ·ðAH�V�öqðNFIZ[ZK=�@ABQ=RL�û�=¶ðCF�úQ=/_`=�úQ=�ZKPRêN=k_ F0ö�PRX¡êCHKZ[=�ó�HK@`aÐóqFQVWñ·ós?Cö�ñ·?Aós=Eñ·ðCFIñ�ê3=�ó·aÐPQósXYV~ñ·ðC=aÐPQZ[ZKPiû'H[@AB´ñ·óqFI@CV·Z[FIñ·H[PQ@CV�÷[%HKóqVwñkLiPQ?Aó�ö�PQXEêAH[ZK=�ó�ö�PQ@>úQ=�óWñqV½ñ·ðA="é´ê3=�@CJ0ëÿFQêAêAZ[H[ö�Fiñ·H[PQ@�HK@GñsPF~XEH[ö�ó·PQõ«ñqFQV·\GH[@AB(aÐPQósX ì iï�÷G<`=�ö�PR@C_¥LRH®ñ�ö�PQ@>úQ=�óWñqVcé´êN=�@CJ0ëÛV·ðCFQó·=k_¡_CFiñsF±H[@Gñ·P~ñsðA='aÐPRó·X@A=�ö�=�VsV·FQó·OTaÐPQó8<`PIaÄñwûUFIós= �±<`J VWO`VWñ·=�XEV�÷�M�Z�V·PCLné´êN=�@CJ0ëíV·ðCFIós=�_�_AFiñqF�XYF�O0H[@Cö�ZK?C_A=VW?AòCó·PR?`ñ·H[@A=�õ�Z[P>ö�FIZ`úiFIósH�FIòAZ[=�V�Liû'ðAH�öqð¡ós=�V·H�_`=UPQ@¡ñ·ðA="êAó·P`ö�=�VsV�VwñqFQöq\]÷Q<GñsFRöq\`V�PQ@EPQ@A='@CP>_A=FIós=�@APQñYúGH�V·HKòAZ[=Tñ·PÿPIñsðA=�ó¶<`PIaÄñwûUFIós=6�±<`J @AP`_`=�V�÷�ø�=�ðCF�úR=T_A=�úQ=�ZKPRêN=k_�Fÿö�PQXEêAH[ZK=�óFIZ[BQPQósHKñ·ðAX�ñsðCFiñ�H[_`=�@Gñ·H#�C=�V�VWðCFQó·=k_!ú�FQó·H�FIòCZK=kV�L¥?CV·H[@AB�HK@Gñs=�ó·õ�êAósP`ö�=k_`?AóqFIZ¬FI@NFIZ[O>V·H�V�L�FQ@C_öqðCFI@ABR=�V�ñ·ðC=�H[ó"_`=�ö�Z[FQósFIñ·H[PQ@¶ñ·PYFI@�=�ô`êAZ[H�ö�HKñ"FIZ[ZKP`ö�FIñ·H[PQ@�HK@�V·ðCFIós=�_TVWêNFQö�=R÷

L<J�ã�Tqä¬ã&Y ?�á´çQJ��� ?.BWæF?âAèwä�á ä�T��OJ\?.B�MOIAIPB·èWçQ?âAèwäcá� Jná´ç��PY ?�ã���@

üu@�êAós=�ú>H[PQ?CV(û�PQós\ ì®î)�iï�L3û�=EXE=�FQV·?Aós=�_0ñ·ðA=Eê3=�ó·aÐPQósXEFQ@Cö�=¡PIa�\Q=�ós@A=�ZêCó·PRBQóqFIXYV�ñsP�=�VWõñ·H[XEFIñ·=¸FQ@�?CêAêN=�ó¶ò3PQ?A@N_¾aÐPRóYñ·ðC=0êN=�óWaÐPRó·XYFQ@Cö�=�PIa8é´ê3=�@CJ0ë¼FQêAêAZ[H[ö�FiñsHKPR@CVEPQ@ PQ?CóVWO`VWñ·=�X�÷`ø!=~FIZ�VWP�?CVW=k_¶XEH�ö�ósPIõ�òN=�@CöqðAXYFIós\`V�ñsP �G?CFQ@Gñ·HKaÐOYñ·ðA=±ê3=�ó·aÐPQósXYFI@Cö�="PIa�VWê3=�ö�H �3öé´êN=�@CJ0ëÆö�PR@CVWñ·ós?Cö1ñqV�÷`üu@Tñ·ðCH[V�V·=�ö�ñ·H[PQ@�L`û�=�_`=kV·ö�ó·H[òN=~FQ@C_�_`H[Vsö�?NV·V�ñsðA=~XE=�FQV·?Aós=�XE=�@GñqVö�FIósósHK=k_�PQ?`ñ±PQ@�ñsðA=¡òCFQV·=�Z[HK@C=8ê3=�ó·aÐPQósXEFQ@Cö�=8PIacós=�FQZ®õuFIêAêCZKH�ö�FIñ·H[PQ@�òN=�@CöqðAXYFIós\`V"PR@0PQ?Có

ö�Z[?CVwñs=�ók÷ �UðC=�<`ë�b��5é(J0ë�J7�� �Aî/V·?AH®ñs=�PQa�òN=�@CöqðAXYFIós\`V ìKîQî�ïUö�PQ@CV·H�VwñqV�PIa´ó·=kFIZ[H[VWñ·H�öé´êN=�@CJ0ë �:FQ@C_%APQó·ñ·óqFI@�FIêCêAZKH�ö�FIñ·H[PQ@CV�÷

�nP"V·?AXEXYFIósH#��=¬PQ?CónXE=�FRVW?Aós=�XE=�@RñqV�L�û�=�@APIñs=�ñ·ðNFiñ�F'@CFIH[úQ=¬ñ·óqFI@CVWaÐPQósXYFiñ·H[PQ@±PQaAó·=kFIZKõH[VWñ·H�ö(é´ê3=�@CJ¸ë¾FIêCêAZKH�ö�FIñ·H[PQ@CVcaÐPQóU<>PIaÄñwûUFIós=A�±<`J6=�ô`=�ö�?Añ·H[PQ@¶_`P>=kVc@APQñ�BRHKúR=�?CV�ñ·ðA=(_`=�õVWH[ó·=k_�ê3=�ó·aÐPQósXYFI@Cö�=Q÷[�UðA=UòCFQV·=�Z[H[@A=�ê3=�ó·aÐPQósXYFI@Cö�=�H�V�H[ZKZ[?CVWñ·óqFiñ·=k_�H[@ %H[BQ?Aós=(îQ÷iø!=�ñsósFQ@CVWõZ[FIñ·=�_TaÐPR?Aó´<`ë�b�� é(J0ëcJ�����Cî %CPQó·ñ·óqFI@TêAósPQBQóqFIXYV��Îø��"ë¬øÆüW<Ab�L3<`øÆüwJ LNJ��±^�ü!�8LM"ëcë ��� �(?CVWH[@AB�PQ?Có�ö�PQXEêAH[Z[=�ó�FI@N_¸ñwû�P0<`ëcbX�ºé(J0ë�J7�� �Aî �S�«M"^ �±L½b����"M�~b��êAó·PRBQóqFIXYV�òGO�ðCFI@N_¥÷Aø!=~=�úiFIZ[?CFiñs=�_¶ñ·ðA=~ê3=�ó·aÐPQósXYFI@Cö�=±PQ@�FEö�PQXEXEP`_`HKñwO¶ö�Z[?CVWñ·=�ó´ö�PQ@AõVWH�VwñsHK@CB¸PIa(ë�=�@Gñ·H[?AXEüWü����H[@G?Aôÿ@CP>_A=�V�Lö�PQ@A@A=kö1ñs=�_Ûú>H�F¸VWñsFQ@C_AFIóq_ b�ñsðA=�ós@A=�ñ¡@A=�ñwû�PRó·\`V�÷�UðA=�V·=�ò3=�@NöqðAXYFIós\(êCó·PRBQóqFIXYVnFQó·=�\G@CPiû'@�ñ·P(=�ô`ðAH[òAHKñBRP>P>_�VWê3=�=k_`?AêCVPR@�V·ðCFIós=�_~XE=�X�õPQósO�V·O`Vwñs=�XYV¡ì®îRî�ï�÷&��Piû�=�úR=�ókLAñsðA=�H[ó±êN=�óWaÐPRó·XYFI@Nö�=�PR@!<>PQaÄñwû�FQó·=,�±<`J VWO`VWñ·=�XYV(VWðCPiû�V_`H 0]=�ós=�@GñYò3=�ðCF�ú>H[PQók÷ %CPQó %APRóWñsósFQ@¾FIêAêCZKH�ö�FIñ·H[PQ@CV�L½ø��´ë¬øÆüW<`b5FQ@C_¾J��±^�ü!��=�ô`ðAH[òAHKñVWê3=�=k_`?AêCV�L�û'ðA=�ós=�FRV�ñsðA=UêN=�óWaÐPRó·XYFI@Nö�=cPQa3<>øÆüwJ FI@C_�M"ë�ë�����_`=�BRósFR_`=�VV·HKBR@AH �3ö�FI@GñsZKORLFQV¥ñsðA=�@G?CX8ò3=�ónPQaAêAósP>ö�=�VsVWPRósV�HK@Cö�ó·=kFQV·=�V�÷�b¬ú>H�_`=�@GñsZKORL�ê3PQó·ñ·H[@AB"û�=�Z[Z®õ�ê3=�ó·aÐPQósX¡H[@AB�V·ðCFIós=�_`õX¡=�XEPQósO8êAósPQBRósFQXEVñsP�<>PQaÄñwû�FQó·=F�±<`J6_`P>=kV¬@APQñc@A=�ö�=�VsV·FQó·H[Z[O8Z[=�FQ_¡ñsP�?A@AHKaÐPQósXEZKO¡BQP>P`_= �/ö�H[=�@Cö�OQ÷¥M+Z[FQó·BR=�êCFQóWñ±PIa¬ñsðAH�V´ê3=�ó·aÐPQósXYFI@Cö�=�_`=�BQóqFQ_AFIñ·H[PQ@0H�V±_`?A=�ñ·P�ñ·ðA=�aÎFQö�ñ±ñ·ðCFIñó·=kFIZKõ�FQêAêAZ[H[ö�Fiñ·H[PQ@(ò3=�@CöqðAXYFQó·\`V�ðNF�úQ=¬ö�PRXEêAZK=�ô´XE=�XEPRó·O±FQö�ö�=�VsV¥êCFIñWñs=�ós@CV�L�û'ðAH�öqð�ö�FI?NVW=kV=�ô`êN=�@CV·HKúR=¡V·ðCFIós=�_¸XE=�XEPQósO0FQö�ñ·H[úGHKñwO�PQ@ <>PQaÄñwû�FQó·= �(<`JÛ÷¥üu@ÿ<>=kö1ñsHKPR@�$NL3û�=Eû'HKZ[Z¬FI@AõFIZ[O���=±ñ·ðA=�ö�FI?NVW=kVUaÐPQó�ê3=�ó·aÐPQósXYFI@Cö�=�_`=�BRósFR_AFiñsHKPR@¶aÐ?CóWñsðA=�ó"FQ@C_�êAósPQê3PRV·=±PQê`ñsHKXEH���FiñsHKPR@ñ·=�öqðC@AH��G?A=kV�aÐPQó´é´ê3=�@CJ¸ë êAó·PRBQóqFIXYV�PQ@�<>PIaÄñwûUFIós=��(<`J V·O`Vwñs=�XYV�÷

� MÆå ?�á"çQJ½å G�I"âAèDY5è�� ?âAèwäcáP@

�UðA=�òCFRVW=�ZKH[@A=Uñ·óqFI@NVWZ�FiñsHKPR@�PQa�<Aëcb��Àé(J0ë�J7�� �Aî�êAó·PRBQóqFIXYV�L�_A=�Vsö�ósHKò3=�_EH[@¶<>=�ö�ñ·H[PQ@ �ALVWðAPiû�V�ñsðCFiñ�ö�PR@>úQ=�ó·ñ·H[@AB¡VWðCFQó·=k_>õ�X¡=�XEPQósO¡êCó·PRBQóqFIXYV�ñsPY<`PIaÄñwûUFIós= �±<`J êAó·PRBQóqFIXYV�ó·=�õ�R?CHKós=�V�PRê`ñ·H[XEH#�kFiñ·H[PQ@n÷Q<>PIaÄñwûUFIós=��±<`J H[X¡êCZK=�X¡=�@GñsFiñsHKPR@CV�ðCF�úR=�V·ðAPiû'@EFQö�ö�=�êAñsFIòCZK=�êN=�óWõaÐPQósXEFQ@Cö�=¶PQ@ \Q=�ó·@C=�Z¬êAósPQBRósFQXYV/ì iï�÷ ��Piû�=�úR=�ókL¥\Q=�ós@A=�Z[V�_`H 0]=�ó8aÐó·PRX ós=�FQZKH�VWñ·H�ö/VWðNFIós=�_X¡=�XEPQósO¸FIêAêCZKH�ö�FIñ·H[PQ@CV±H[@!ñwû�P�=kV·V·=�@Gñ·H�FIZ�ûUF�O`V��.�Wî)�(H[@�ñs=�ósXYV~PIa'VWðNFIós=�_!_AFiñqFTFRö�ö�=�VsVêCFiñ·ñ·=�ós@CV'FQ@C_9� ���H[@Tñ·=�ósXYV'PIa½ñ·ðA=�V·H���=±PIañ·ðA=�V·ðCFQó·=k_�_AFIñsFEV·êCFQö�=Q÷MÆó·=kFIZ[H[VWñ·H�ö'FIêCêAZKH�ö�FIñ·H[PQ@EñwO>êAH�ö�FQZKZ[O�ö�PQ@CV·H[VWñsV¬PIa�VW=�úQ=�ósFQZCFIZ[BQPRó·HKñ·ðAXYVñsðCFiñUFQö�ö�=�VsV�ñ·ðC=

VWðCFQó·=k_ _AFiñqF HK@:_AH 0]=�ós=�@Gñ¶ûUF�O`V�÷��UðC=�V·=0FQö�ö�=�VsVYêCFIñWñ·=�ó·@NV/XYF�O¾ós=�V·?AZKñ¶HK@ÆH[@Cö�ó·=kFQV·=�_X¡=kV·VsFIBR=Uñ·óqF��/ö'H[@Eñ·ðA=´?A@C_`=�ó·Z[O>HK@CB�<>PIaÄñwûUFIós=F�±<`J�Z�F�OQ=�ókLiû'ðAH�öqð/H�V¬=�ô`ê3=�@CV·H[úQ=UaÐósPQX5FêN=�óWaÐPRó·XYFI@Nö�='ú>HK=�û'êNPRHK@Gñk÷��±=�ós@A=�ZCêAósPQBRósFQXEV�_AP�@APIñ�=�ô`ðAH[òAH®ñ�ñ·ðA=´ö�PQXEêAZ[=�ôYFQö�ö�=kV·V�êNFiñWõñ·=�ós@CVUPQanaÐ?AZ[Z®õuV·H#��=�_TFIêAêAZ[H�ö�FiñsHKPR@CVUFI@C_¶ñ·ð>?CV'_AP¡@CPIñ'òAósHK@AB¡PQ?Añ�ñsðA=�V·=~FQ_A_AH®ñsHKPR@CFIZ�ö�PRVWñsV�÷<>=�ö�PQ@C_`Z[OQL>FQV¬ñ·ðA=(VWH���="PIanV·ðCFIós=�_Y_AFIñsF~H�V�HK@Cö�ó·=kFQV·=�_¥LQû�=´PQòCV·=�ósúQ=k_8ñsðCFiñ�ñ·ðA=(ö�PRðA=�ós=�@Cö�=FI@C_!?Aê]_AFiñs=Yñ·óqF��/öYH[@Cö�ó·=kFQV·=�_!VWH[BQ@AH#�Nö�FI@Gñ·Z[OQ÷ ��O>êAH�ö�FIZcó·=kFIZ[H[VWñ·H�öEVWðCFQó·=k_!X¡=�XEPQósO�FIêAõêAZKH�ö�FIñ·H[PQ@CV�L>VW?CöqðTFQVcñ·ðA=~<`ë�b��Æé(J0ë�J7�� �Aî�FIêCêAZKH�ö�FIñ·H[PQ@CV�LGXEF�OEðCF�úR="_AFIñsF8V·=�ñqV�ñ·ðCFIñFIós=(H[@�ñ·ðA=~PRós_`=�ó'PIaBQH[BRFIò>OGñ·=R÷

�nPÿFQ_A_Aó·=kV·V8ñ·ðA=�FQòNPiúR=�H�V·V·?A=kV�L�û�=�ðCF�úR=�H[XEêAZ[=�XE=�@Gñ·=k_¾PRê`ñ·H[XEH#�kFiñsHKPR@CV�ñ·ðCFIñ¡aÎFQZKZHK@Gñ·P¡ñsðAó·=�=�ö�FIñ·=�BRPQósHK=kV�÷

� ��PQXEêA?`ñqFiñsHKPR@Tós=�êNFIó·ñ·HKñ·H[PQ@AH[@AB�aÐPQó'Z[P>ö�FIZ[H®ñwO¶=�@AðCFQ@Cö�=�XE=�@Gñ� ë�FIBR=(F�ûUFIós="PRê`ñ·H[XEH#�kFiñ·H[PQ@Tñs=�öqðA@AH��G?A=�V� <>ðCFQó·=k_�_AFIñsFEV·êCFQö�=(ós=�_A?Cö1ñsHKPR@¶ñsðAó·PR?ABQðTêAósHKúiFIñ·H���FiñsHKPR@

!$OMP PARALLEL DO DO k=2, nz-1 DO i=ist, iend DO j=jst, jend DO m=1, 5 rsd(m,i,j,k)=... ... ENDDO ENDDO ENDDO ENDDO ...!$OMP END PARALLEL DO

!$OMP PARALLEL DO DO j=jst, jend DO i=ist, iend DO k=2, nz-1 DO m=1, 5 rtmp(m,k)=rsd(m,i,j,k)-... ... ENDDO ENDDO ENDDO ENDDO ...!$OMP END PARALLEL DO

!$OMP PARALLEL DO k=2, nz-1 DO i=ist, iend!$OMP DO DO j=jst, jend DO m=1, 5 rsd(m,i,j,k)=... ... ENDDO ENDDO!$OMP ENDDO ENDDO ENDDO ...!$OMP END PARALLEL DO

!$OMP PARALLEL DO DO j=jst, jend DO i=ist, iend DO k=2, nz-1 DO m=1, 5 rtmp(m,k)=rsd(m,i,j,k)-... ... ENDDO ENDDO ENDDO ENDDO ...!$OMP END PARALLEL

(a) original code (b) after computation repartitioning

� � � ª��Cª {hk|~}R�RrWx�rwv®h�y��coq}>x�turwv�rwv®h�yGvpyG�Q�>�u�R²Rtwhk�QrwvpyGo����'e~l[twhk|  �½� �N�

��� �� ��������������� ������ ����!"�"�#�"�� �������$ë�FIBQ=�õ�òNFQV·=�_Æ<>PIaÄñwûUFIós=7�±<`J V·O`Vwñs=�XYV�HKXEêAZ[=�XE=�@Gñ�ö�PR@CVWH�VWñ·=�@Nö�OÀòGO =�ô`öqðNFI@ABRHK@AB¾HK@`õaÐPQósXEFIñ·H[PQ@ÿFiñ±ñ·ðC=¡êNFIBQ=EZ[=�úR=�Z�÷�S�=�ñwû�=�=�@!VWO>@CöqðAósPQ@CH#�kFiñ·H[PQ@�êNPRHK@GñqV�L]ñsðA=YêCFIó·ñ·H�ö�H[êCFIñ·H[@AB@AP`_`=�V8=�ôAöqðCFI@CBQ=EHK@AaÐPQósXEFIñ·H[PQ@ FIò3PQ?Añ8û'ðAH�öqð @AP`_`=�V8û'ó·PQñ·=YH[@RñsP0=�FRöqðÿêCFIBR=Q÷nM¼@CP>_A=ñ·ðCFIñ¶û'ó·HKñ·=kVEñ·PÛFÛêCFIBR=TH[@:V·ðCFIós=�_ÀXE=�XEPQósO ñsðG?NV/ò3=�ö�PRXE=�VEñsðA=�ñ·=�XEêNPRósFQó·O&%('*)�+-,PIa¬ñsðCFiñ±êCFIó·ñ·H�ö�?CZ[FQó´êCFQBQ=R÷3M êCFIBR=�ö�PQ?AZ�_0ðCF�úR=�X8?CZ®ñsHKêAZ[=¡ñ·=�X¡ê3PQóqFIósO�%.'*)�+-,0/"HKa¬ñ·ðC=�ós=FIós=�X�?AZ®ñsHKêCZK=�@AP`_`=kVEû'ó·HKñ·H[@AB�ñ·P�ñ·ðA=0VsFIXE=TêCFIBR=�ò3=�ñwû�=�=�@�V·OG@NöqðAó·PR@AH���FiñsHKPR@Ûê3PQH[@RñqV�÷�UðA=�ûUF�O8ñsðAH[VcPiû'@A=�óqV·ðAHKêYöqðCFQ@ABQ=kV¬_`?AósH[@AB~ñ·ðC="êAósPQBRósFQX XYF�O¡V·HKBR@AH#�Nö�FQ@RñsZKO¡F[0]=�ö�ñcñ·ðC==�ô`=�ö�?`ñ·H[PQ@�ñ·H[XE=±aÐPQó'ñsðA=~FIêAêAZ[H�ö�FiñsHKPR@�÷

%APQó�=�ôAFIXEêAZ[=QLQñ·ðA=´XYFIH[@/ZKP>PRêYHK@¶M´ëcë ���:ö�PQ@GñsFQHK@CV�VW=�úQ=�@YêCFIóqFIZ[ZK=�Z&�~é"õ�Z[P>PQêCV�÷RM"ZKZñ·ðA=kVW=YêCFQósFQZKZ[=�Z��~é"õ�ZKP>PQêNV~FQö�ö�=�VsV~F�VWðCFQó·=k_�FQó·óqF�O�,0/-13254�6#786 906;:0<i÷.%HKúR=YPIa�ñ·ðA=kVW=/V·=�úR=�@êCFIóqFIZ[ZK=�Z �~é"õ�Z[P>PQêCVnêCFIó·ñ·HKñ·H[PQ@�FIósósF�O�,0/=1¬?CV·HK@AB"ñ·ðA=�PQ?Añ·=�ó½XEPRVWñ½H[@C_`=�ô�:I÷Qé´@A=cZ[P>PQê�_`P>=�VòAZKP`öq\�êCFIó·ñ·HKñ·H[PQ@AH[@ABNLI?CV·HK@AB~ò3PIñsð>7`FI@C_?9�H[@C_`H�ö�=�VcFI@C_¡ñsðA=�PIñsðA=�ó�êNFIó·ñ·HKñ·H[PQ@CV�?NVWH[@AB~ñsðA=*9HK@C_A=�ô¥÷ �Uð>?CV�LIñsðA=�XYFIH[@YZKP>PRê¡PQa�M"ëcë ��� ðCFRV�aÐPQ?Có�FQö�ö�=kV·V�êCFIñWñs=�ós@/öqðCFI@ABR=�V�ê3=�óc=�úQ=�ó·OZKP>PQê¾H®ñs=�óqFiñ·H[PQ@n÷ %H[BQ?Aós=0HKZ[ZK?NVwñsósFIñ·=�V8ñ·ðC=TöqðCFQ@ABQ=¶HK@ÀFQö�ö�=�VsV�êCFIñWñs=�ós@CV8ò3=�ñwû�=�=�@Ûñwû�PPIa¬ñsðA=�V·=�Z[P>PQêCV�÷Q�UðAH[V±FRö�ö�=kV·V´êCFIñWñ·=�ó·@!öqðCFI@CBQ=8û'H[Z[ZH[@Cö�?Aó~F/Z�FIósBQ=8@>?AX�òN=�ó±PIacós=�XEPIñs=@AP`_`=/ós=5�G?A=�VWñsV±H[@ÿñ·ðC=¶VW=kö�PR@C_!êCFIóqFIZ[ZK=�Z�ZKP>PRê�÷ �½P0F�úQPQH�_�HK@A=��/ö�H[=�@Gñ�FQö�ö�=kV·V±êCFIñWñs=�ós@CV�Lñ·ðA=UêAósPQBRósFQX @C=�=�_CVñ·P±òN=�V·=�Z[=�ö�ñ·H[úQ=UFIò3PQ?`ñ�û'ðAH�öqð¡@AP`_`=�VñsPQ?Cöqð�û'ðAH[öqð¡ê3PQó·ñ·H[PQ@CV�PQaCñ·ðC=_AFiñqFA÷+%CPQó"=�ôAFIXEêAZ[=QL`ñsðA=8ö�P`_`=8XEF�O�ðCF�úQ=�FYö�PQ@CV·H�Vwñs=�@Gñ"FQö�ö�=�VsV'êCFIñWñs=�ós@0FQö�ósPRVsVUZ[PGPRêCV�L

H®a�ñsðA=(HK@A@C=�ó 9�õ�Z[P>PQê/H�V�êCFIó·ñ·HKñ·H[PQ@A=k_YH[@CVwñs=�FR_/PIa¥ñsðA=(PQ?`ñs=�ósX¡PGVwñ :qõ�Z[PGPRê/H[@/ñ·ðC=A�CóqVWñ�Z[PGPRê@A=�VWñ�÷+%H[BQ?Aós= �Ðò&�UV·ðAPiû�V�ñ·ðC=~ó·=kVW?AZKñ·H[@ABYö�P`_`=~FIaÄñ·=�ó � %(4���������7�%() , +����., � 7���7�%() 7 )��Q÷

�UðAH[V�PRê`ñ·H[XEH#�kFiñsHKPR@�ós=5�G?AH[ó·=kVñ·ðA='ö�PQXEêAH[ZK=�ó54 V�FIòAH[Z[H®ñwO~ñsP�_`=�ñs=�ö�ñ�aÐ?AóWñsðA=�ó�êNFIóqFIZ[ZK=�ZKH�VWXHK@EñsðA="Z[P>PQêE@A=�VWñ�÷Rø�=´?CV·=�_¡ñ·ðC="ë�PQZ�FIósH[V�êNFIóqFIZ[ZK=�ZKH���H[@AB~ö�PRX¡êCHKZ[=�ócaÐPQó�ñsðAH[VcêA?AósêNPGVW=�ì®î �ï�÷��Piû�=�úQ=�ó�L½@APIñ/FQZKZUZ[PGPRêÛ@A=kVwñqVYFIZ[ZKPiû5ñsðAH[V¡PRê`ñ·H[XEH#�kFiñ·H[PQ@�òN=kö�FQ?CVW=�V·PQXE=�H[@A@A=�óEZKP>PQêNVö�FI@C@APIñEòN=�êCFQósFQZKZ[=�Z[H���=�_�÷ø!=�FIêAêCZKH[=�_�úiFIósHKPR?CV�ñs=�öqðA@CH��G?A=kV�L¬V·?Cöqð�FRVEFQ_A_`H[@AB!ó·=k_`?A@`õ_AFI@Gñ�ö�PQXEêA?`ñqFiñ·H[PQ@nL¥ñ·P0=�@CFIòCZK= � %.4���������� 7�%.)�, +����(, ��7�� 7�%.) 7 )��Yñ·ðAósPQ?ABRðAPQ?Añ~ñ·ðA=/û'ðCPQZ[=êAó·PRBQóqFIX�÷

���� � ��$ ����� �3!.��� ������� ��� ������ �� �

!$OMP PARALLEL DO DO 200 J=1,N DO 200 I=1,M UNEW(I+1,J) = ... VNEW(I,J+1) = ... PNEW(I,J) = ... 200 CONTINUE!$OMP END PARALLEL DO

DO 210 J=1, N UNEW(1,J) = UNEW(M+1,J) VNEW(M+1,J+1) = VNEW(1,J+1) PNEW(M+1,J) = PNEW(1,J) 210 CONTINUE

!$OMP PARALLEL DO DO 200 J=1,N DO 200 I=1,M UNEW(I+1,J) = ... VNEW(I,J+1) = ... PNEW(I,J) = ... 200 CONTINUE!$OMP END PARALLEL DO

!$OMP PARALLEL DO DO 210 J=1, N UNEW(1,J) = UNEW(M+1,J) VNEW(M+1,J+1) = VNEW(1,J+1) PNEW(M+1,J) = PNEW(1,J) 210 CONTINUE!$OMP END PARALLEL DO

(a)original code (b)after page aware optimization

����� ª"!Cª � x��ko�Âc��x�two´°�}Gv�rw|~vp¯1x�rwv®h�yN�Q�u�G²Qtwhk�Qrwv®yRo"{�Â��3{��"l[twh�|+ei� �u 

ë�FIBQ=�õ�F�ûUFIós=(PQê`ñsHKXEH���FiñsHKPR@CV(?CV·=8ñsðA=E\G@CPiû'ZK=k_`BQ=8ñ·ðCFIñ´ñsðA=/<>PIaÄñwûUFIós=,�±<`J XYFQHK@`õñsFIH[@CV�ö�PQðA=�ó·=�@Cö�=�Fiñ�ñ·ðA=UêCFQBQ=�BQóqFI@>?AZ�FIósH®ñwOR÷�ø!=Uû'HKZ[ZA_`=kV·ö�ó·H[òN=�ñwû�P"ñwO>ê3=�VPIa����#� +"$%�.'&�(, +%'��� 7 4�7)(#���7�%()�/1÷+%H[ósVWñ�L>û�=�ñ·óqFI@NVwaÐPRó·X¼F8V·ðCFQó·=k_/FIósósF�O¡ò>O*��� 1 1.7 )+�ILNVWP8ñ·ðCFIñ'FQó·óqF�O8êNFIó·õñ·HKñ·H[PQ@AH[@AB FQö�ó·PGV·V8@AP`_`=�V¡êAZ�FQö�=�V�ñ·ðA=�êCFQóWñsH®ñsHKPR@CVEFiñEêCFIBR=�ò3PQ?C@C_AFIósH[=�V�÷ %CPQó¡=�ôAFIXEêAZ[=QLö�PQ@NVWH�_`=�ó�FI@/FIósóqF�O-, 254�6#786 906;:0<"PQa�VWH���=., 20/ 6213.6213.62140<i÷+�UðC=´FIósóqF�O~ñwO>êN=�H�Vc_APQ?AòAZ[="êCó·=�õö�H�VWH[PQ@ �«VWH���=�PIa3F´_`PR?AòAZ[=�êCó·=kö�H�VWH[PQ@�@>?AX�òN=�ó½H�V /"ò>OGñ·=kVnH[@¡PQ?Aó�VWO`VWñ·=�X ��÷�ü�aAñ·ðA='ö�PQXEêAH[ZK=�óêCFIó·ñ·HKñ·H[PQ@CVcñ·ðCH[V�FIósósF�O5,�?CV·H[@AB�ñsðA=´H[@C_`=�ô 9¬HK@/ñ·ðA=(êCFIóqFIZ[Z[=�ZCós=�BRHKPR@�LIñsðA=�@�êCFQ_C_`HK@CB�ñ·ðC=�CósVWñ"FI@N_¶ñsðA=�V·=�ö�PR@C_T_`HKXE=�@CVWH[PQ@�PQa*,0û'H[ZKZnêAó·P`_`?Nö�=�, 2�6(621 7362183 621840<i÷�M'aÄñs=�ó�êCFR_A_`H[@ABCLñ·ðA=¸V·H���=�PIa(ñ·ðC=¸VWðNFIós=�_ÀFIósósF�O9, H[@Cö�ó·=kFQV·=�VEVWZ[H[BQðGñ·Z[OQ÷��"Piû�=�úQ=�ó�L�ñsðA=0ò3PQ?A@C_CFIósHK=kVEPIaêCFIó·ñ·HKñ·H[PQ@A=k_0FIósóqF�O�öqð>?A@A\`V(FIós=�@APiû FIZ[H[BQ@A=k_�û'H®ñsð0ñsðA=�êNFIBQ=8òNPR?A@C_AFQó·H[=�V�÷:�UðCH[V´PQê`ñsH®õX¡H���FIñ·H[PQ@Yós=�_`?Nö�=�V¬aÎFIZ�VW=(VWðNFIósHK@AB�FQó·PR?A@C_�ñ·ðC="ò3PQ?A@C_CFIósHK=kV¬PIa�êNFIó·ñ·HKñ·H[PQ@A=k_YVWðCFQó·=k_E_AFiñqFöqðG?C@A\`V�÷

�UðA="V·=�ö�PR@C_:���#� +"$%�.'&�(, + %;��� 7 4�7)(<��� 7�%.)E_`=kFIZ�V¬û'HKñ·ðYñsðA=�êCFQBQ=�V·ðCFIê3=Q÷Rüu@¶F~ö�PRZK?AXE@`õXEF��wPQó�Z[FQ@ABQ?CFQBQ=�V·?Cöqð¶FRV�%APRóWñsósFQ@�LRF8êAósP>ö�=�VsV�ñsðCFiñ�û'ó·HKñ·=kV�F�ö�PQZ[?AXE@/H[@¶F~ñwû�P8_`H[XE=�@`õVWH[PQ@CFQZ�FIósósF�OTû'HKZ[Z�ñ·PR?Cöqð¸X�?Cöqð0aÐ=�û�=�ó±êCFIBR=�V"ñ·ðCFQ@!F�êAósP>ö�=�VsV´ñ·ðCFIñ(û'ósH®ñs=�V±FTó·Piû~÷]M"V

FI@¶=�ôAFIXEêAZ[=QL>ZK=�ñ��Æò3=~F iõ �íFIósóqF�O�PQa½VWH���=¡î5��[$Qô]î)� �$±FQ@C_/HKñsVU=�ZK=�XE=�@GñsVUFIós=A$¡ò>OGñ·=�VHK@Gñ·=�BQ=�ósV�÷Aü�a�ñ·ðC=�V·H#��=�PIa�ñ·ðC=8êCFQBQ=~H[@�<>PIaÄñwûUFIós= �±<`J H�VF$ �~S(L`ñsðA=�@0=kFQöqð0ö�PQZ[?AXE@�PIa� ö�FI@�ò3=~XYFIêAê3=�_TñsP/F�êNFIBQ=R÷+�Uð>?CV�L>ñ·ðA=�ó·=�FQó·=Yî)� [$¡êCFQBQ=�VUP`ö�ö�?Aê>O>HK@CBEö�PQósó·=kVWê3PQ@N_>õHK@ABÿî)� [$Tö�PRZK?AXE@CV±H[@��±÷¥ü�a'F�@CP>_A=¡û'ósHKñ·=�V~FTö�PRZK?CX¡@!H[@��~L]ñ·ðA=�@�PR@AZ[O0PQ@A=EêCFQBQ=¡H[VF[0]=�ö1ñs=�_¥÷Ué´@¾ñsðA=�PIñ·ðC=�óYðCFQ@C_¥L�û'ósH®ñsHK@AB FÿV·HK@ABRZK=�ósPiû H[@�� ñsPQ?CöqðA=kVEFIZ[ZUñ·ðA=�êCFQBQ=kVPiû'@A=�_¶òGO/FQZKZ3ñsðA=±êCFIó·ñ·H�ö�H[êCFiñsHK@CB�@AP`_`=kV�÷ �UðAH�V'Vsö�=�@NFIósHKP�H�VUH[ZKZ[?CVWñ·óqFiñ·=k_/H[@%H[BQ?Aós=2�A÷>üu@ñ·ðAH�V2�CBR?Aó·=RL]ñ·ðA=�ó·=YH�V�FQ@Ûé´ê3=�@NJ0ë êCFIóqFIZ[ZK=�Z�ZKP>PRê¸aÐPRZKZ[Piû�=k_¸ò>O!FTV·=�ósH�FIZ�Z[PGPRê�÷�üu@!ñ·ðC=êCFIóqFIZ[ZK=�ZCZ[P>PQê�LR=�FQöqð¶@AP`_`="û'ósH®ñs=�V¬ñ·P8HKñsV�êCFQóWñsH®ñsHKPR@A=�_EòAZ[P`öq\`VcPQa¥ñ·ðA=(VWðCFQó·=k_YFIósósF�O`V�FQ@C_ñ·ð>?CVcñ·=�XEê3PQóqFIósHKZ[OEPiû'@CV�ñsðA=(êCFIBR=�V¬H[@/HKñsV�êCFQóWñsH®ñsHKPR@�÷��UðC=�@YñsðA=´XYFRVwñs=�óc@CP>_A=(ö�PQêAH[=�V�FVWH[@ABQZ[=~ósPiû ñsP¶FI@APQñ·ðA=�ó�ósPiû:aÐPQó´=�FRöqð�V·ðCFQó·=k_TFQó·óqF�OYH[@�ñ·ðA=8V·=�ósH�FIZ�Z[P>PQê0FI@N_�H[@�=10]=�ö�ñ�Lñ·PQ?NöqðA=�V(FIZ[Zñ·ðA=¡êCFIBR=�V"aÐPQó´ñ·ðA=kVW=EV·ðCFIós=�_0FQó·óqF�O`V�÷3<>?AòCV·=5�G?A=�@RñsZKORL]û'ðA=�@¸ñ·ðA=kVW=EVWðNFIós=�_FIósósF�O`VcFQó·=(ós=�FQ_/òGO/ñ·ðA=~öqðAH[Z[_¶@AP`_`=�V�L`=�FRöqð�öqðAH[Z�_/@AP`_`=±ðCFRVcñsPEó·=)�R?C=�VWñ�?Aê]_AFIñ·=�V�aÐósPQXñ·ðA=�XYFRVwñs=�ó"@CP>_A=Q÷&�UðCH[V"HK@Cö�?AóqV�VW?CòCVwñqFI@Gñ·H�FIZ½PiúQ=�ó·ðA=kFQ_¥L`û'ðCH[öqð0ö�FI@�òN=¡F�úQPRH[_`=k_¶HKa�ñ·ðC=VW=kö�PQ@N_�ZKP>PQê�H�V�=�ô>=kö�?`ñs=�_�H[@0êCFQósFQZKZ[=�Z�÷Aüu@�ñ·ðC=�PQósHKBRHK@NFIZ�ò3=�@CöqðCXEFQó·\Tö�P`_`=RLAñ·ðC=�V·=�ö�PR@C_ZKP>PQê�H[V±FTVW=�ó·H�FIZZ[PGPRê�L]=�úR=�@0ñsðAPQ?CBQð¸HKñ�ö�FQ@0òN=EêCFQósFQZKZ[=�Z[H���=�_�÷:�UðAH�V±H�V´ò3=�ö�FI?CV·=¡êCFIóqFIZKõZK=�ZKH���H[@AB¸F0V·XYFIZ[ZcZ[P>PQêÿH[V8@APIñEFIZ[û�F�O`V±êAósP��CñsFIòCZK=YH[@¾V·ðCFIós=�_!XE=�XEPQósO¸êCó·PRBQóqFIXEXEHK@ABN÷�UðG?NV�LGñ·ðCH[VUPRê`ñ·H[XEH#�kFiñ·H[PQ@¶ðAHKBRðAZ[HKBRðRñqVUF�_`H#0]=�ós=�@Cö�=´PQanPQê`ñsHKXEH���FIñ·H[PQ@�VWñ·óqFiñs=�BRO¡ò3=�ñwû�=�=�@VWðCFQó·=k_�FQ@C_T_`H[VWñ·ósH[òA?`ñ·=k_TX¡=�XEPQósO/=�@>ú>HKósPQ@CX¡=�@GñsV�÷

���� �?!.����������� �3���� � � ���"��� � � �3�"�� ���UðAH[V8PQê`ñsHKXEH���FIñ·H[PQ@ÿH[V8FIH[X¡=k_ Fiñ�ós=�_`?Nö�H[@AB�ñ·ðA=�VWH���=YPQa�ñsðA=¶V·ðCFIós=�_ÿ_AFIñsF�V·êCFQö�=Eñ·ðCFIñX8?CVWñ~òN=YXYFI@NFIBQ=k_¸ò>O�ñsðA=/<>PIaÄñwûUFIós= �(<`J V·O`Vwñs=�X�÷në�PIñs=�@Gñ·H�FIZ[Z[OQL¥FIZ[Z�úiFIósH[FQòAZK=kV´ñ·ðCFIñFIós=Yó·=kFQ_!FI@C_ÿû'ósH®ñ·ñ·=�@!û'HKñ·ðCHK@ÛêNFIóqFIZ[ZK=�Z�ó·=�BQH[PQ@CV~FQó·=/V·ðCFIós=�_�úiFIósH[FQòAZ[=�V�FQ@C_!X8?CVWñ8ò3==�ô`êAZ[H[ö�H®ñsZKOY_`=kö�Z�FIós=�_YFQV�VW?Nöqð�÷ �"Piû�=�úQ=�ókLIû�=�ðCF�úR='aÐPQ?A@N_Eñ·ðCFIñ�F�V·H[BQ@AH#�Nö�FQ@Gñc@>?AX8ò3=�ó�PIaVW?CöqðYúiFQó·H�FIòAZ[=�VcFIós=�Iós=�FR_>õ�PR@AZ[O�(û'HKñ·ðAH[@YñsðA="êCFQósFQZKZ[=�ZCó·=�BQH[PQ@CV�÷�%A?CóWñsðA=�ósXEPQós=QLQû�=�ðCF�úR=PQòCV·=�ósúQ=k_/ñsðCFiñkLAaÐPQó�ö�=�ó·ñsFQHK@�V·ðCFIós=�_�FIósóqF�O>V�L>_`H 0]=�ós=�@Gñ"@CP>_A=�V'ós=�FR_TFQ@C_Tû'ó·HKñ·=�_AH[V��wPQH[@GñêCFIó·ñsV½PIa`ñsðA=�FQó·óqF�OQ÷�ø�=�ó·=�aÐ=�ó½ñ·P�ñsðA=�V·=cúiFQó·H�FIòAZ[=�V½FQV�/ 7 )���� + $ %('*)�+-,n_AFIñsFA÷küu@�ñsðA=�ö�PQ@Gñ·=�ô>ñPIaCñsðA=�é´êN=�@CJ0ë!êAósPQBQóqFIX�L�ñ·ðA=kVW='FIós=�V·ðCFQó·=k_�úiFIósH[FQòAZ[=�V�÷)�"Piû�=�úQ=�ókLkHK@�ñ·ðA='ö�PQ@Gñ·=�ôGñ�PIa3F<>PIaÄñwûUFIós=A�±<`J6H[XEêAZK=�XE=�@GñsFIñ·H[PQ@�L>H[@CVwñqFI@Cö�=�V�PIa�ò3PIñsð�ö�FQ@/ò3=´êAósH[ú�FIñ·H���=k_Yû'H®ñsð�ö�=�ó·ñsFIH[@êAó·=kö�FQ?`ñ·H[PQ@CV�÷C�UðA= �CósVWñUòN=�@A=1�Añ'PQanêAósHKúiFiñsH#�kFiñsHKPR@¶Vwñs=�XYV�aÐósPQX+ñ·ðA=(aÎFRö1ñUñ·ðCFIñUFRö�ö�=�VsV¬ñsPF¶êAósH[ú�FIñ·=¡úiFIósH[FQòAZK=¡H�V"ñwO>êAH�ö�FQZKZ[O�aÎFQVWñ·=�ó(ñ·ðCFQ@!FRö�ö�=kV·V"ñ·P�FTVWðNFIós=�_0úiFQó·H�FIòAZ[=QL3=�úR=�@�H®aUFZKP`ö�FQZKZ[O�ö�FQöqðC=�_�ö�PRê>O�PQa�ñ·ðA=¡VWðCFQó·=k_�úiFIósH[FQòAZK=8H[V(F�úiFIH[Z[FQòAZK=R÷&�UðAH�V�H�V´òN=kö�FI?NVW=¡FQö�ö�=�VsVW=kVñ·P0V·ðCFIós=�_�úiFIósH[FQòAZK=kV±@A=�=k_�ñ·P�ñ·ósH[BQBQ=�ó~ö�=�ó·ñsFQHK@ÿö�PRðA=�ós=�@Nö�O¸FQ@C_!òNP>PQ\>\R=�=�êCHK@AB�FRö1ñ·H[PQ@NVû'H®ñsðAHK@0ñ·ðA=E<>PIaÄñwûUFIós= �(<AJ ÷:�UðC=8VW=kö�PR@C_�HKXEê3PQó·ñsFQ@Rñ´òN=�@A=1�Añ(PIa¬êAósH[ú�FIñ·H���FIñ·H[PQ@�H[V"ñ·ðC==10]=�ö1ñPQ@~=�ZKH[XEHK@CFIñ·H[@AB�aÎFQZ[V·=cV·ðCFIósH[@ABC÷ �UðC=cPiúQ=�ósFQZKZIö�PQðA=�ó·=�@Cö�O´PiúQ=�ó·ðA=kFQ_(H�VnFIZ�V·P'ó·=k_`?Cö�=k_òN=kö�FI?NVW=~ö�PRðA=�ós=�@Nö�O/ðCFRV�ñ·PEò3=~@APiû XYFIH[@GñsFQHK@A=k_/aÐPRó�FYVWXYFQZKZ[=�ó'V·ðCFIós=�_T_AFiñqFEVWH���=R÷

�OJ @>æABwâ�@

ø�=ùFIêCêAZKH[=�_ ñ·ðC=í_A=�Vsö�ósHKò3=�_ PQê`ñsHKXEH���FIñ·H[PQ@CVÀò>O ðCFQ@C_ ñsP¼VW=�úQ=�óqFIZTPIa�ñ·ðA= <`ë�b��é(J0ëcJ7����Cî'òN=�@CöqðAXYFIós\`V�FQ@C_¶FRöqðAHK=�úQ=k_YXYFIós\Q=�_Eê3=�ó·aÐPQósXYFI@Cö�="H[XEêAó·PiúR=�XE=�@GñqV�÷�%½H[BIõ?Aó·= $�V·ðAPiû�Vcñ·ðA=P�C@CFQZ]êN=�óWaÐPRó·XYFI@Nö�="PRò`ñsFQHK@C=�_Yò>OEñ·ðC=´òCFRVW=�ZKH[@A=´ñ·óqFI@CV·Z�Fiñ·H[PQ@�FI@C_/V·?Aò`õVW=)�R?C=�@Gñ´PRê`ñ·H[XEH#�kFiñ·H[PQ@NV�÷Nø�=�êAó·=kVW=�@Gñ"ñ·ðC=�V·êN=�=�_`?AêNV�aÐPRó�aÐPQ?Có %CPQó·ñ·óqFI@�ö�P`_`=kV �Îø��"ë�õøÆüW<`b�LN<`øÆüwJ LCJ��±^'ü!��LCM´ëcë�� �P�UFQ@C_/ñwû�P �Æò3=�@NöqðAXYFIós\>V2�«b ���"M �~b�LAM´^ �F��÷

0

1

2

3

4

5

6

1 2 4 8 1 2 4 8 1 2 4 8 1 2 4 8 1 2 4 8 1 2 4 8

Baseline�Performance Optimized�Performance

wupwise swim mgrid artequakeapplu

����� ª��3ª � ostul[h�tw|�x�yRfqo h�l¡l�hk�Qt¸þGh�turutWx�y �k�¾x�yGz r��h:{ ²`oqyRf·gR|�x�tw¿I�Tl[twh�| rwgGoÿe � m{°'  �  Y�����Q�"²`oqyRf·gR|�x�tw¿~�u�Gv�rwo'h�y��k�`�I�R�R�Gx�yGzE�(|�x�fWgGvpyRoq�q�

üu@�PR@A=�PIa(ñ·ðA= %CPQó·ñ·óqFI@�ö�P`_`=�V�L�ø��"ëcøÆüW<`b�L�ñsðA=�òCFRVW=�ZKH[@A=Tñ·óqFI@CV·Z�Fiñ·H[PQ@ÀFIZ[ó·=kFQ_`OðCFQ_�FQö�ö�=�ê`ñqFIòAZ[=´V·ê3=�=�_A?Aê�LAFI@N_¶V·P8û�=(_`H�_�@APIñ'FIêAêCZKO¡aÐ?Aó·ñ·ðA=�ó'PQê`ñsHKXEH���FIñ·H[PQ@CV�÷�%CPQó�ñ·ðC=ñ·ðAós=�=�PIñsðA=�ó.%APRóWñsósFQ@8ö�P`_`=kV�LI<>øÆüwJÛLIJ��±^'ü!��LIFI@C_�M"ëcë ���8Lkû�=�PQò`ñqFIH[@A=�_�VWH[BQ@AH#�Nö�FI@GñêN=�óWaÐPRó·XYFI@Nö�=�H[X¡êCó·PiúR=�XE=�@GñsV]û'HKñ·ð � %(4���������7�%() , +���., � 7�� 7�%.) 7 )+�cFQ@C_ ���"� + $%�('&�., + %;��� 7�$4�7)(<��� 7�%.) /1÷�üu@8<>øÆüwJÛL�FIêCêAZKH�ö�FIñ·H[PQ@±PIa>ñsðA= ���"� + $%�('&�., + %;��� 7 4�7)(<���7�%()(V·ðAPiû'@(HK@2�CBR?Aós=��HKXEêAósPiúQ=k_¶ñsðA=8ê3=�ó·aÐPQósXYFI@Cö�=�_`óqFIXYFiñsH[ö�FIZ[ZKOR÷Aø�=¡VWZ[H[BQðGñ·Z[O¶=�@AðCFI@Nö�=�_TñsðA=8ê3=�ó·aÐPQósXYFI@Cö�=PIa�J��±^'ü!� òGOÿFIêCêAZKO>H[@AB � %.4���������� 7�%.)�, +����(, ��7�� 7�%.) 7 )��Q÷M´ëcë�� ��L�û'ðCH[öqðÛV·ðAPiû�V±PQ@C=PIa�ñ·ðC=�XEPRVWñ(ö�PQXEêAZ[=�ô0FQö�ö�=kV·V"êCFiñ·ñ·=�ós@CV(FIXEPQ@CBYñ·ðA= %APRóWñsósFQ@0FIêAêAZ[H�ö�FiñsHKPR@CV�LNðCFRV"ò3=�=�@PQê`ñsHKXEH���=�_T?CV·H[@ABEòNPQñ·ð � %.4���������� 7�%.) , +���., � 7�� 7�%.) 7 )+��FQ@C_ ���#� +"$%�.'&�(, + %'��� 7 4�7)(#���7�%()�/1÷�UðA=±òCFQV·=�Z[HK@C=(PQaM´ëcë�� � êN=�óWaÐPRó·XYV�òN=�ñWñs=�ó'û'HKñ·ðTñwû�P�ñ·ðNFI@�û'H®ñsð¶aÐPR?Aó'êAósP`ö�=�VsV·PQóqV�÷Güu@ñ·ðAH�V±FIêAêAZ[H�ö�FiñsHKPR@�LNñ·ðA=EVWðNFIós=�_0FQó·óqF�O ,0/-13254�6#786 906;:0<¡H[V(òAZ[P`öq\�êCFIó·ñ·HKñ·H[PQ@A=k_�?CV·HK@CB¶ò3PIñ·ð 7FI@C_ 9"HK@C_AH[ö�=�V(VWPYñsðCFiñ�,0/-1�H�V"êNFIó·ñ·HKñ·H[PQ@A=k_0FQö�ö�PQóq_`H[@ABEñ·P¶ñ·ðA=�HK@N_`=�ô 9'PR@�ñwû�P¶êAó·P`ö�=�VWõ

VWPRósV�LNFQ@C_0H�V�aÐ?CóWñsðA=�ó(êCFIó·ñ·HKñ·H[PQ@C=�_0?NVWH[@AB/ñsðA=¡HK@C_A=�ô�7�PQ@¸aÐPQ?Aó(êAósP`ö�=�VsV·PQóqV�LCFQ@C_0FQBRFIH[@?CVWH[@AB�ñ·ðA=EH[@C_`=�ô 9"PQ@�=�H[BQðGñ±êAó·P`ö�=�VsVWPRósV´FI@C_�VWPTPQ@�÷¥ë�FQóWñsH®ñsHKPR@AHK@CB/?CV·HK@AB�ñ·ðC=¡H[@C_`=�ô 7ó·=kVW?AZKñsV�HK@�X8?CZ®ñsHKêAZ[=�@AP`_`=kV'û'ósH®ñsHK@ABEñsP/FYVWH[@ABRZK=±êCFQBQ=�FI@N_¶ñsð>?CV�ö�FQ?CV·=�V�aÎFIZ�V·=�VWðCFQó·H[@ABN÷�UðA=�ós=�aÐPRó·=RLAû'ðA=�@C=�úQ=�ó ,0/-1~H�V�êNFIó·ñ·HKñ·H[PQ@A=k_�FQö�ö�PQóq_`H[@ABEñ·P¶ñ·ðA=�H[@A@A=�ó´H[@C_`=�ô 7ÄL]HKñ´V·? 03=�ósVF0ê3=�ó·aÐPQósXEFQ@Cö�=�_`ó·PRêÛPiû'H[@AB�ñ·P¸ñ·ðA=TH[@Cö�ós=�FRVW=k_!aÎFIZ�VW=TVWðCFQó·H[@ABN÷<>H[@Cö�=¶ñ·ðA=TPQêAñ·H[X¡H���=k_úQ=�óqV·HKPR@¸aÐPRó8M"ë�ë����¼FQZKûUF�O`V(êCFIó·ñ·HKñ·H[PQ@CV ,0/=1¡?NVWH[@AB�ñ·ðC=/HK@N_`=�ô 91LnHKñ�@APIñ�PQ@AZ[O�F�úRPQH�_AVñ·ðAH�V'H[@Cö�PR@CVWH�VWñ·=�@Gñ�V·êN=�=�_`?AênLAòA?`ñ"FQZ[V·PYVWðAPiû�V�X�?Cöqð�òN=�ñWñ·=�ó�PiúQ=�ósFQZKZ3ê3=�ó·aÐPQósXEFQ@Cö�=R÷

%APQó�ñ·ðC=�� FQêAêAZ[H[ö�FiñsHKPR@CV�LGM"^ � V·ðAPiû�=�_EH[XEêAó·PiúR=�_Yê3=�ó·aÐPQósXYFI@Cö�="FiaÄñs=�óUêAósHKúiFIñ·H���H[@ABVW=�úQ=�óqFIZIFQó·óqF�O`VNñ·ðCFIñnû�=�ó·=�@APQñ½_`=kö�Z�FIós=�_±FQV¥êCó·H[úiFiñ·=�H[@±ñ·ðC=¬PQósHKBRHK@NFIZIö�P`_`=R÷�üu@�b ���´M�~b"Lû�=�_`=�ósH[úQ=�_¾òN=�@A=1�Cñ¡aÐósPQX XYFQ\GH[@ABÿö�=�óWñqFIH[@¾êCFQósFQZKZ[=�ZUZ[PGPRêCV¡_`O>@CFIXEH�ö�FIZ[Z[OÛV·öqðA=k_`?AZ[=�_¥Lñ·ðAPR?ABQð�ñ·ðA=~PRó·H[BQH[@CFQZ�é´ê3=�@NJ0ëÆ_`H[ó·=kö1ñ·H[úQ=kVUVWê3=�ö�H �C=k_�VwñqFiñ·H�ö~VsöqðA=�_`?CZKH[@ABC÷

�UðA=~F�úQ=�óqFIBR=�òCFRVW=�ZKH[@A=~VWê3=�=k_`?AêTPIa�VWHKô�FIêCêAZKH�ö�FIñ·H[PQ@CV�H�V±îQ÷ /�*8FI@C_¶ñ·ðC=±F�úR=�óqFIBR="PQêAõñ·H[X¡H���=k_TêN=�óWaÐPRó·XYFI@Nö�=±H[VA�A÷[î *�PQ@ /¡êAó·P`ö�=�VsVWPRósV�÷��Uð>?CV�LAPR?Aó'êAósPQê3PRV·=�_/PRê`ñ·H[XEH#�kFiñsHKPR@CV�L� %(4���������7�%() , +���., � 7�� 7�%.) 7 )+�IL ���"� +"$%�.'&�., + %'����7 4�7)(<��� 7�%.) /qL¬FQ@C_ � �'� +=/0/ ��, 7������ 7)(<��� 7�%.)ó·=kVW?AZKñ¸H[@ºF�úQ=�ósFQBQ= *[��� ê3=�ó·aÐPQósXYFI@Cö�= HKXEêAósPiúQ=�XE=�@Gñ¸PQ@ùPR?Aó!<Aëcb�� é(J0ë�J7�� �AîòN=�@CöqðAXYFIós\`V�÷

� ��ä�á"ç BWæP@>èwäcáP@

üu@/ñ·ðCH[V�êCFIê3=�ókLGû�=´ðCF�úQ="_A=�Vsö�ósHKò3=�_¶PQ?Aó�=�ô`êN=�ó·H[=�@Nö�=�V�û'HKñ·ð/ñsðA=´FQ?`ñ·PRXYFiñ·H�ö'ñsósFQ@CV·Z[FIñ·H[PQ@FI@C_�aÐ?AóWñsðA=�óðCFQ@C_>õ�PQê`ñsHKXEH���FiñsHKPR@�PIaNó·=kFIZ[H[VWñ·H�ö�é´ê3=�@CJ¸ë�FQêAêAZ[H[ö�Fiñ·H[PQ@NV�PQ@¡F"ö�PQXEXEP>_AH®ñwOö�Z[?CVwñs=�ó�PQa�û�PRó·\`VWñsFiñsHKPR@CV�÷`ø�=�ðNF�úQ=�_`=�X¡PR@CVWñ·óqFiñ·=k_�ñ·ðCFIñ�LAaÐPRó'ñ·ðC=�V·=8FIêAêCZKH�ö�FIñ·H[PQ@CV�L`ñ·ðC=é´êN=�@CJ0ë:êCó·PRBQóqFIXEXEHK@ABYêNFIóqFQ_`H[BQX XYF�OTò3=¡=�ô>ñ·=�@C_`=�_0ñ·PT_`H[VWñ·ósH[òA?`ñ·=k_¸V·O`Vwñs=�XYV�÷3ø!=ðCF�úQ=¡_`H[Vsö�?NV·V·=�_�H[VsV·?A=�V±FIósH�VWH[@AB¶H[@�ñ·ðA=YFI?Añ·PQXYFIñ·H�ö�ñsósFQ@CVWZ�FiñsHKPR@0PIa�é´êN=�@CJ0ë FIêAêCZKH�ö�FIõñ·H[PQ@CV�aÐPQó�<>PQaÄñwû�FQó·=P�(<`JÛ÷`ø�=±ðCF�úR=�ñ·ðC=�@TêAó·=kVW=�@Gñ·=�_¶VW=�úQ=�óqFIZNêAósPQBQóqFIX5PRê`ñ·H[XEH#�kFiñ·H[PQ@NVaÐPQóTêCFIBR=�õ�òCFQV·=�_:<>PQaÄñwû�FQó·=-�(<AJ V·O>VWñ·=�XYV�÷Uüu@ PR?AóTêN=�óWaÐPRó·XYFI@Nö�=�VWñ·?N_`HK=kV�LUû�=¸ðCF�úR=aÐPQ?A@C_¶ñ·ðNFiñ�F¡òCFQV·=�Z[HK@C=(ö�PQXEêAH[ZK=�ó�ñsósFQ@CV·Z[FIñ·H[PQ@�L`V·HKXEH[Z[FQó�ñsP¡ñ·ðA=±PQ@C=´?CV·=�_¶aÐPQó"é´ê3=�@CJ¸ëPQ@�<`J0ë XYFQöqðAH[@A=kV�L`O>H[=�Z�_AV'VWê3=�=k_`?AêCV'aÐPQó�V·PQXE=±PIanñsðA=�ö�P`_`=kVUòA?`ñ�?A@NFQö�ö�=�ê`ñqFIòAZ[=±êN=�óWõaÐPQósXEFQ@Cö�=EaÐPRó�PIñsðA=�óqV�÷nM'aÄñs=�ó�FQêAêAZ[O>HK@AB0ñ·ðA=/êCó·PRêNPGVW=k_�PRê`ñ·H[XEH#�kFiñsHKPR@CV±õ � %(4���������7�%(), +���., � 7�� 7�%.) 7 )+�IL����#� +"$%�.'&�(, + %'����7 4�7)(<��� 7�%.)�/1L3FI@C_-� �;� +=/ / � , 7�� ���7)(<��� 7�%.)/õcû�=±PRòCVW=�ó·úR=�_VWH[BQ@AH#�Nö�FI@Gñ~HKXEêAósPiúQ=�X¡=�@GñsV´HK@ÿê3=�ó·aÐPQósXEFQ@Cö�=R÷]üu@�ñsðA=Y@A=�ôGñ~êAðNFQV·=¡PQa�PR?Aó~êAósP��w=�ö�ñ�L]û�=HK@Gñ·=�@C_!ñ·P�?NVW=Y=�ô>êCZKH�ö�HKñ�XE=�VsV·FQBQ=�õ�êCFRV·V·HK@CB¶H[@Ûö�PQ@��w?A@Cö1ñsHKPR@ÿû'H®ñsð¾<>PIaÄñwûUFIós= �±<`J FQ@C_HK@>úQ=kVwñsHKBGFiñs="ñsðA=~=10]=�ö�ñsV'PQaPR?AóUPQê`ñsHKXEH���FiñsHKPR@CV'H[@¶ñsðAH�V'ð>OGòCó·H�_�FQêAêAósPRFRöqð�÷

� J T Jnã�Jná"çQJ @

�k�"°�}`osy`  � þRh�tw�G|E���u°�}`osy`  � �i � twhk}`h��uo1z±��yGzR�G��rutu�(eirWx�y>zRx�tWz´Â � �3l�h�t�eIg>x�twoqz~ ¡oq|±�h�tu� � twh���tWx�|~|~vpyR�R� ��³3osf·gC� �coq}N�p�G°�fsr1�C�1�����I�

�I�"e`�»���x�tw¿�xkzRx�� � �'oqjpoqgRost �´��N� ������x1µux�|~hkyi���À���½� {���Â�|~¯1xI�Â�� �n�¬{h1¹!x�yGz�À� �>��x�oqyGos}`hioqjÄ����³Ntwo1x�zR|�x�tw¿I�q�3eQgGx�two1z¶|~oq|~h�tu�Efqhk|~}R�RrwvpyG�8hky�yRosr��h�tw¿I�'h�l��h�tw¿i���rWx�rwvphkyG�q� �����������nÈ�É�ÊGËQÌ«Ó·ÒW�>�khkjÄ�`���Q�GyGhR�>�I�R}G}C�A�1���I���I�AþGoq²Qtw�>x�tu�/�q�k���I�

�Q� �"�Q�N�C���´�G{�� ���N�Qx�yGz8� ���>��x�oqyGoq}`hiosjÐ����°�}`oqy`  � hky�yGosr«��h�tw¿~h�l]��h�tw¿I��rWx�rwvphkyR�q� ��vpy� ÒuÈ1Ç���È����GË�ÊRÓ·Ò�ÇWÈ�É�Ê>ËIÌÄÏ�ÑiÔ! "$#k�]�q�k���Q�

�R�" ¡v�rw�u�RgGvp�wx¶eRx�rwhQ�� ¡h�rwhky>x�twv ��v�tWx�yGhR�!�h��ugGvph/³]x�yGx�¿�xQ�]x�y>z�eRx�rwh��ugGv¬eIoq¿Ivp�k�GfWgGvÄ�%�u°�|±�yRv � � {��> ��twvKz�� � { þ>x�fqvpjpv�r��8l�h�t'{jp�G��rwo·tUx�y>z ��jph�²>x�j]{hk|~}R�RrwvpyG�±vpyE°�}`oqy`  � � ��vpy

� ÒuÈ1Ç��NÈ���Ì��GÓ��¡È�Ò���Ø��RÈWʶÈ�Ñ��]ÊRÓ·Ñ ��� Ê�Ê� ÏÄÇWÍ�ÌÄÏÐÈ�ÑGØ�Í�Ñ>Þ��`È1È� Ø������� ��� �����������1���k�Gj�����k�I�k�

�I� ���A{½twh1�oqjpj �]�C{��o·rWx�yGh��Ivpf �Q�3�Ux�tutwv®�'{��`��oqjp�uh�y��Q� � v�twfq�wx�¿`� � �C{½tWx�vp��x�yGz�{��A°"!AyGost1���m�¹IrwoqyGzRvpyG��°�}`oqy>  � l�h�t]���' E¸ Ex�f·gRv®yRoq�q� �´vpy � Ò�È1Ç��`È��½Ì��GÓ��������$# � �% �GË�ÊRÓ·ÒuÇwÈ�É(ÕÊGËQÌÄÏ[ÑIÔ! ��������&�'UÏpÔ�� � Ó·Ò �qÈ�ÒWÉ~Í�Ñ>Ç·Ó)("Ó·ÌÄÖ½È�Ò*��Ï�ÑIÔ"Í�Ñ`Þ �nÈ�É�Ê>ËIÌÄÏ�ÑIÔ �nÈ�Ñ �qÓ·ÒwÓ·Ñ`ÇWÓ�� � �� �������+�����h���oq|(²`o·tc�����k�Q�

�Q�-,"�¬eIf·gI�G��rwo·tEx�y>z »(�c ¡vpjpoq�q� �«»Uvp��rutwvp²G�Rrwoqz�°�}`osy`  � �¬m¥¹QrwosyG�uvphkyR��rwh�°�}`oqy`  � l[h�teR  � {jp�R��rwostw�q� � vpy � ÒuÈ1Ç��'È��¡Ì��GÓ.�EÈ�Ò*��Ø��RÈ·ÊÀÈ�Ñ/�]ÊRÓ·Ñ �0� Ê�Ê� ÏÐÇwÍ�ÌÄÏÄÈ�ÑGØEÍ�Ñ`Þ.�`È1È� Ø���1�� �2� � ���������1�3�k�Rj��Y���k�k�I�

�I��³�� e`�¥Â�²AzQoqj�tWx�gR|�x�y�x�y>z�³�� �"�3��h�yG�Q�%��{hk|~}Gvpjpost´�u�G}R}`h�tur�l�h�t±zRx�rWxYzRvp��rutwvp²G�Qrwvphky�h�y���' EÂÀ|´�Gj�rwvp}Rtwhifqos�u�uh�tw�q� �54IÈ�ËQÒWÑ`Í6 ¥È�� �GË�ÊRÓ·ÒuÇwÈ�É�Ê>ËIÌÄÏ�ÑIÔ��C��h�jÐ�N���i�`yGhQ�>�R�G}G}C� ���k��� ���I���hif·r"�q�k���Q�

�Q� ��vp�kg � ostul�h�tw|�x�yGfsoUþGh�turutWx�y�þGh�tw�R|E� ����vp�kg � ostul�h�tw|�x�yRfqoUþGh�turutWx�y8j®x�yG���>x��koU�u}`oqfsv �Gf1x��rwvph�yN�R��ostw�uvphkyT��� �Q� ��³3oqfWgN� �coq}N�>{�� � {½�«³ �����k���k�i�3��h��G��rwhkyC�>³Nos¹A�®�C�1�����I�

�Q�" �� � hih�rwgEx�y>z ±�R ¡v®�uos��xkzQoq�q���u ¡vpfstwh�rWx��u¿IvpyG�Q�`ÂÛ��os�ÿ��xq�~rwh ��x�twyGos�u�c ¡�Gj�rwvp}Rtwhifqoq����uh�tw�q� � �¥ÒuÍ�Ý �%�QÍ�ÑGÑ`Ó7 ØW�R}G}C�`��� �I�k�I�3�1�k���Q�

�q�Q��¬�ihky � x��u�R|�x�jpjpvp¿A�¬eQoq�RyG���8��x�vU ¡vpyC��x�yGz ���GzRh�jpl'mnvp�kosyG|�x�yRyN� ��³3h1��x�tWzR�E°�}`oqy`  �o·¹Rosfq�Rrwvphky�hkyE�uh�l[r«��x�two´zRvp��rutwvp²G�Qrwo1z��ugGx�two1zE|~os|~h�tu�����Q��rwoq|~�q� �Tvpy �WÑRÌ� 9�EÈ�Ò*��Ø��RÈ·Ê0È�Ñ�3ÊRÓ·Ñ� � & ��:·ÊRÓ·ÒWÏÎÓ·Ñ`ÇWÓ·Ø�Í�Ñ>Þ �WÉ�Ê� ®Ó·É�Ó·ÑRÌÎÍ�ÌÄÏÐÈ�ÑRØ������� � ��� �;�<�1�� Ex1�ÿ�������i�c�Noqfsrw�Qtwo��h�rwoq�¬vpyY{h�|~}G�Qrwost�eQfqvpoqyRfqok�>���k�k�I�AeQ}RtwvpyR�kost9,nostwj®x��Q�

���k� �c�>zQhkj�l�mnvp��oqyG|�x�yGy ��twoq� �Ux�osturwyRost±��os�uj®o·� � ���khkyRoq�=,�v®�ugGx�j¬Âc�uj®h�r1�½ Ex�¹0»Uh�|~oqvp¿kxx�y>z � hIzRh � x�tWx�zQ�i���ue � m½{hk|~}C�¥Â ��os� � oqyRf·gR|�x�tw¿�eQ�Rvprwo´l�h�t± ¡o1x��u�RtwvpyG� � x�tWx�jpj®osj{h�|~}G�Qrwost � o·tul�h�tw|�x�yGfqo�� � vpy � ÒuÈ1Ç��´È �>�1�� �2� �?�������;@A�¡È�Ò*��Ø*�QÈ·Ê:È�Ñ?�]ÊRÓ·Ñ� �� Ê�Ê� ÏÄÇWÍ�ÌÄÏÐÈ�ÑGØ"Í�Ñ`Þ��>È1È� Ø�@CB]ÓuÇsÌÄËIÒwÓ�('È�Ì«Ó·Ø�Ï�Ñ �nÈ�É�Ê>ËIÌ«Ó·Ò �`ÇsÏÎÓ·Ñ>ÇWÓ�@D������ER�F�k�Rjp�Y�����Q�k�A}R}N�� �>�q�Q�

�1�I�U�ÿv®jpjpv®x�| � jp�G|~o�� ��x�|~h�y�»�h�x�jpjphR�����GzRh�jpl�mnvp��oqyG|�x�yGyN���kh�gGy ��twh��Rr1�%��xq����hio��Gv®yR�kost1�³�gRhk|�x��T�Nx1��twosyGfqok�-��x�o�µ�vpy �Coqok�'»Ux��Iv®z � x�zR�>xI���½�GyGgRoq�GyR� � x�oq¿`� � vpjpj � h�rurwoqyR�kost1��Nxq��twoqyGfqo ��x��Rf·gi��o·tw�kost1�Ix�y>z � osyG��³3�C� � � x�tWx�jpjpoqj`}Rtwh���tWx�|~|~vpyR���¬v�rwg � hkj®x�twvp�q� � ��������nÈ�É�Ê>ËIÌ«Ó·ÒW�>}G}C�G�����i���i�`»Uoqfqos|(²`ost'�q�k���I�