Triangle Counting with Cyclic Distributions - NSF PAR

Triangle Counting with Cyclic DistributionsAndrew Lumsdaine∗†, Luke Dalessandro‡, Kevin Deweese∗, Jesun Firoz†, Scott McMillan§

∗University of Washington, †Pacific Northwest National Lab, ‡Indiana University, §CMU/Software Engineering Institute∗{al75, deweeskg}@uw.edu, †{andrew.lumsdaine, jesun.firoz}@pnnl.gov, ‡[email protected], §[email protected]

Abstract—Triangles are the simplest non-trivial subgraphs andtriangle counting is used in a number of different applications.The order in which vertices are processed in triangle countingstrongly effects the amount of work that needs to be done (andthus the overall performance). Ordering vertices by degree hasbeen shown to be one particularly effective ordering approach.However, for graphs with skewed degree distributions (such aspower-law graphs), ordering by degree effects the distribution ofwork; parallelization must account for this distribution in orderto balance work among workers. In this paper we provide an in-depth analysis of the ramifications of degree-based ordering onparallel triangle counting. We present approach for partitioningwork in triangle counting, based on cyclic distribution and somesurprisingly simple C++ implementations. Experimental resultsdemonstrate the effectiveness of our approach, particularly forpower-law (and social network) graphs.

I. INTRODUCTION

At the 2017 HPEC conference, one of this paper’s authors(AL) presented a simple C++ implementation of paralleltriangle counting to demonstrate the capabilities of the C++standard library and its parallelization mechanisms (e.g., asyn-chronous tasking). A challenge was issued to the communityto develop a better performing, but similarly concise, im-plementation. The presented algorithm incorporated featuresthat, when combined with degree-based ordering, results ina very fast simple implementation that is quite competitivewhen compared to highly optimized shared memory imple-mentations (notably for graphs with highly skewed degreedistributions), as will be shown in our contribution (TeamNWGraph) to an upcoming paper in IISWC [1].

With the challenge algorithm as a jumping-off point, thispaper takes a deeper look at what algorithmic and imple-mentation details are necessary to obtain high performancein parallel triangle counting. We focus on the following: thetriangle counting algorithm, the graph representation, orderingthe graph by degree, efficient set intersection, and effectiveparallel load balancing. Although triangle counting is a fairlywell-studied algorithm, much of the development and rigor-ous study has focused on sequential complexity. In addition,

∗Partially supported by NSF SI2-SSE Award 1716828 and DOE PSAAP

II Award DE-NA0002377†Funding for this work was partially provided by the High Performance

Data Analytics (HPDA) program at Pacific Northwest National Laboratory.This research was also supported by PNNL’s Segmented Global AddressSpace (SGAS) LDRD Initiative.‡

Partially supported by DOE PSAAP II Award DE-NA0002377§This material is based upon work funded and supported by the Department

of Defense under Contract No. FA8702-15-D-0002 with Carnegie MellonUniversity for the operation of the Software Engineering Institute, a federallyfunded research and development center. [DM20-0658]

v1<latexit sha1_base64="L/0JIBe9OYFCCCQ2Ju6OP8PDaKI=">AAACs3icbVHbahRBEK0db3HjJdFHXwZDwKcwI3h5EWIk6GNENwnsDkt1T81uk74M3TWLy7Cf4Kuif+bf2LMbJbvJgYbDqUufqhK1VoGz7E8vuXX7zt17W/f72w8ePnq8s/vkNLjGSxpIp50/FxhIK0sDVqzpvPaERmg6ExcfuvjZjHxQzn7leU2FwYlVlZLIUfoyG+fjnb3sIFsivU7yS7J3uP3xF0ScjHd7v0elk40hy1JjCMM8q7lo0bOSmhb9UROoRnmBExpGatFQKNql10W6H5UyrZyPz3K6VK9WtGhCmBsRMw3yNGzGOvGm2LDh6m3RKls3TFauPqoanbJLu8HTUnmSrOeRoPQqek3lFD1KjutZ6yRRTuldhTpQ0Rplmcq1oVpeuhbu27pM7JxeqUcUJ6Qjmih7bGfKO9tt61+zdiS6yJU+i/3+6H0VnRzb8sYCsuVGOkvRdHcXHv38v814zXzzdtfJ6cuDPPLP8ayvYYUteAbP4QXk8AYO4ROcwAAkTOA7/ICfyatkmIikXKUmvcuap7CGxPwFmoTd8A==</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="lYAi3ezFm8u7WvA2S2wLievrc8A=">AAACs3icbVFdSxtBFJ1sa7Vprdo+9mUxCH2S3UK1LwWrCH201KiQLOHO7N1kcD6WmbvBsOQn9FXBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebH7a2dz5eeFs5gV1hlXVXHDwqabBLkhRelQ5Bc4WX/PqkyV+O0XlpzTlNSsw0DI0spAAK1J/xIB1sd5L9ZBbxc5AuQIct4myw07rv51ZUGg0JBd730qSkrAZHUiictvuVxxLENQyxF6ABjT6rZ1qn8V5g8riwLjxD8Yx92lGD9n6ieajUQCO/mmvIl3K9iorvWS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2mSADHCHwUoj1mtpSHMl5aqaaaa25tlGslaNWePMWyIxziU5tSMpbOmuda/YXWfN5knc6Z77f7PIig5NfmLDWjylXISvGp85w7c5L/M4Ga66t1zcPF1Pw34d9I5Olj4usE+s132haXskB2xX+yMdZlgQ/aX3bK76FvUi3iUz0uj1qLnE1uKSD8CG5PcxQ==</latexit>

v0<latexit sha1_base64="Q8LhTgq2XZFE4HtHmlyZMh7OSMI=">AAACs3icbVHbahRBEK0db3HjJdFHXwZDwKcwI3h5EWIk6GNENwnsDkt1T81uk74M3TWLy7Cf4Kuif+bf2LMbJbvJgYbDqUufqhK1VoGz7E8vuXX7zt17W/f72w8ePnq8s/vkNLjGSxpIp50/FxhIK0sDVqzpvPaERmg6ExcfuvjZjHxQzn7leU2FwYlVlZLIUfoyG2fjnb3sIFsivU7yS7J3uP3xF0ScjHd7v0elk40hy1JjCMM8q7lo0bOSmhb9UROoRnmBExpGatFQKNql10W6H5UyrZyPz3K6VK9WtGhCmBsRMw3yNGzGOvGm2LDh6m3RKls3TFauPqoanbJLu8HTUnmSrOeRoPQqek3lFD1KjutZ6yRRTuldhTpQ0Rplmcq1oVpeuhbu27pM7JxeqUcUJ6Qjmih7bGfKO9tt61+zdiS6yJU+i/3+6H0VnRzb8sYCsuVGOkvRdHcXHv38v814zXzzdtfJ6cuDPPLP8ayvYYUteAbP4QXk8AYO4ROcwAAkTOA7/ICfyatkmIikXKUmvcuap7CGxPwFmDfd7w==</latexit><latexit sha1_base64="PrlHVV656P4zfW7QT9l3xLSYw00=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDZLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwLu34s=</latexit><latexit sha1_base64="PrlHVV656P4zfW7QT9l3xLSYw00=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDZLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwLu34s=</latexit><latexit sha1_base64="7deWJVqK5ntJufBtGFxEaq6CnHc=">AAACs3icbVFdSxtBFJ1sa7Vprdo+9mUxCH2S3UK1LwWrCH201KiQLOHO7N1kcD6WmbvBsOQn9FXBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebH7a2dz5eeFs5gV1hlXVXHDwqabBLkhRelQ5Bc4WX/PqkyV+O0XlpzTlNSsw0DI0spAAK1J/xIBlsd5L9ZBbxc5AuQIct4myw07rv51ZUGg0JBd730qSkrAZHUiictvuVxxLENQyxF6ABjT6rZ1qn8V5g8riwLjxD8Yx92lGD9n6ieajUQCO/mmvIl3K9iorvWS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2mSADHCHwUoj1mtpSHMl5aqaaaa25tlGslaNWePMWyIxziU5tSMpbOmuda/YXWfN5knc6Z77f7PIig5NfmLDWjylXISvGp85w7c5L/M4Ga66t1zcPF1Pw34d9I5Olj4usE+s132haXskB2xX+yMdZlgQ/aX3bK76FvUi3iUz0uj1qLnE1uKSD8CGUbcxA==</latexit>

v2<latexit sha1_base64="uI42NKCAe4OS5kNZhOyma/2PGqQ=">AAACs3icbVFdSxtBFL1Z22pjW7U+9mVRhD7JrlD1RbAWaR8tGhWSJczM3k0G52OZuRsMS35CXy3tP/PfdDaxYqIHBg7nfsy59/JSSU9Jct+Kll69frO88ra9+u79h7X1jY+X3lZOYEdYZd01Zx6VNNghSQqvS4dMc4VX/OZbE78aofPSmgsal5hpNjCykIJRkM5H/b3++naym0wRPyfpA9k+Xv3+BwLO+hutv73cikqjIaGY9900KSmrmSMpFE7avcpjycQNG2A3UMM0+qyeep3EO0HJ48K68AzFU/VpRc2092PNQ6ZmNPSLsUZ8KdatqDjMamnKitCI2UdFpWKycTN4nEuHgtQ4ECacDF5jMWSOCQrrmeskmBjiUcGUx6zW0hDmc0PVNHXN7e28jGStmqknGCbEExxIc2pG0lnTbOt/s7rHm8iTPpOddu9rEZycmvzFAjT5QjoJXjV354658aPNcM108XbPyeXebhr4z3DWfZhhBT7BFnyGFA7gGH7AGXRAwAB+wR38jr5E3YhH+Sw1aj3UbMIcIv0PnNHd8Q==</latexit><latexit sha1_base64="yG2vXeWQ2205ul1mtENnuXL9ls8=">AAACs3icbVFbSxtBFJ5sL2rsRdtHXxZF6JPsCrV9KahF7KOlRgPJEs7Mnk0G57LMnA2GJT/B15b6f/oj+m+cTWwx0QMDH9+5zHfOx0slPSXJ31b07PmLlyura+31V6/fvN3YfHfhbeUEdoRV1nU5eFTSYIckKeyWDkFzhZf86muTvxyj89Kac5qUmGkYGllIARSoH+PB/mBjJ9lLZhE/Buk92DlcP/299aebnw02W7f93IpKoyGhwPtempSU1eBICoXTdr/yWIK4giH2AjSg0Wf1TOs03g1MHhfWhWconrEPO2rQ3k80D5UaaOSXcw35VK5XUfE5q6UpK0Ij5h8VlYrJxs3icS4dClKTAEA4GbTGYgQOBIXzLEwSIEb4pQDlMau1NIT5wlI1zVRze71II1mr5uwxhg3xGIfSnJixdNY01/o3rO7zJvNgznS33T8qgpITkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GF/t7acDfg60HbB6rbIttsw8sZZ/YIfvGzliHCTZkN+wn+xV9jHoRj/J5adS673nPFiLSdweI340=</latexit><latexit sha1_base64="yG2vXeWQ2205ul1mtENnuXL9ls8=">AAACs3icbVFbSxtBFJ5sL2rsRdtHXxZF6JPsCrV9KahF7KOlRgPJEs7Mnk0G57LMnA2GJT/B15b6f/oj+m+cTWwx0QMDH9+5zHfOx0slPSXJ31b07PmLlyura+31V6/fvN3YfHfhbeUEdoRV1nU5eFTSYIckKeyWDkFzhZf86muTvxyj89Kac5qUmGkYGllIARSoH+PB/mBjJ9lLZhE/Buk92DlcP/299aebnw02W7f93IpKoyGhwPtempSU1eBICoXTdr/yWIK4giH2AjSg0Wf1TOs03g1MHhfWhWconrEPO2rQ3k80D5UaaOSXcw35VK5XUfE5q6UpK0Ij5h8VlYrJxs3icS4dClKTAEA4GbTGYgQOBIXzLEwSIEb4pQDlMau1NIT5wlI1zVRze71II1mr5uwxhg3xGIfSnJixdNY01/o3rO7zJvNgznS33T8qgpITkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GF/t7acDfg60HbB6rbIttsw8sZZ/YIfvGzliHCTZkN+wn+xV9jHoRj/J5adS673nPFiLSdweI340=</latexit><latexit sha1_base64="MjLAA/D4mJHcDb+fAYM3oV+o0oQ=">AAACs3icbVFdSxtBFJ1sbWtjW7U+9mUxCD7JrtDWl4JaBB+VNiokS7gzezcZnI9l5m4wLPkJfW3Bf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrerX2+s3b9XftjfcfPm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNb9oUmKmYWhkIQVQoH6OB4eDrU5ykMwifg7SBeiwRVwMtlv3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4LTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAj/F6A8pjVWhrCfGmpmmaqub1bppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme61+6fFEHJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6vDgzTgy6Rz/HXh6zr7zHbZPkvZN3bMztkF6zLBhuw3+8P+Rl+iXsSjfF4atRY9O2wpIv0IHeDcxg==</latexit>

v3<latexit sha1_base64="RzPDJ/4k9X3yvo4gaftTYrUDVhM=">AAACs3icbVFdSxtBFJ1s61esVutjX5aK0CfZbbH6IqhF2keljQrJEu7M3k0G52OZuRsMS35CX1vsP+u/6Wxii4keGDic+zHn3stLJT0lyZ9W9OLl0vLK6lp7/dXG5uut7TdX3lZOYEdYZd0NB49KGuyQJIU3pUPQXOE1v/3cxK9H6Ly05juNS8w0DIwspAAK0rdR/2N/azfZT6aIn5L0geyerH+5ZwEX/e3W715uRaXRkFDgfTdNSspqcCSFwkm7V3ksQdzCALuBGtDos3rqdRLvBSWPC+vCMxRP1ccVNWjvx5qHTA009IuxRnwu1q2oOMpqacqK0IjZR0WlYrJxM3icS4eC1DgQEE4Gr7EYggNBYT1znQSIIR4XoDxmtZaGMJ8bqqapa27v5mUka9VMPcMwIZ7hQJpzM5LOmmZb/5rVPd5EHvWZ7LV7p0Vwcm7yZwvQ5AvpJHjV3J07cOP/NsM108XbPSVXH/bTwC/DWT+xGVbZW/aOvWcpO2Qn7Cu7YB0m2ID9YD/Zr+gg6kY8ymepUeuhZofNIdJ/AZ8e3fI=</latexit><latexit sha1_base64="qbHl8fe1v7m498qMUNXoKJkSM9Y=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/KrlL1pVArRR9bNG0gWcKd2bvJ0PlYZu4Gw5Kf4KtS/48/wn/jbFKlSXth4HDux5x7Dy+V9JQkf1rRk6crq2vrG+3NZ1vbz3d2X1x5WzmBHWGVdV0OHpU02CFJCrulQ9Bc4TW/+djkr8fovLTmK01KzDQMjSykAArUl/HgzWDnIDlKZhE/BOkdODjd/HS7/7ubXwx2W7/6uRWVRkNCgfe9NCkpq8GRFAqn7X7lsQRxA0PsBWhAo8/qmdZpfBiYPC6sC89QPGPvd9SgvZ9oHio10Mgv5xrysVyvouJ9VktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwpADlMau1NIT5wlI1zVRz+22RRrJWzdkzDBviGQ6lOTdj6axprvVvWN3nTebenOlhu/+hCErOTf5oA5p8qZwErxrfuQM3+S8zuJkue/cQXL0+SgO+DLa+ZfNYZ/vsJXvFUvaOnbLP7IJ1mGBD9p39YD+j46gX8Sifl0atu549thCR/gsJ1d+O</latexit><latexit sha1_base64="qbHl8fe1v7m498qMUNXoKJkSM9Y=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/KrlL1pVArRR9bNG0gWcKd2bvJ0PlYZu4Gw5Kf4KtS/48/wn/jbFKlSXth4HDux5x7Dy+V9JQkf1rRk6crq2vrG+3NZ1vbz3d2X1x5WzmBHWGVdV0OHpU02CFJCrulQ9Bc4TW/+djkr8fovLTmK01KzDQMjSykAArUl/HgzWDnIDlKZhE/BOkdODjd/HS7/7ubXwx2W7/6uRWVRkNCgfe9NCkpq8GRFAqn7X7lsQRxA0PsBWhAo8/qmdZpfBiYPC6sC89QPGPvd9SgvZ9oHio10Mgv5xrysVyvouJ9VktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwpADlMau1NIT5wlI1zVRz+22RRrJWzdkzDBviGQ6lOTdj6axprvVvWN3nTebenOlhu/+hCErOTf5oA5p8qZwErxrfuQM3+S8zuJkue/cQXL0+SgO+DLa+ZfNYZ/vsJXvFUvaOnbLP7IJ1mGBD9p39YD+j46gX8Sifl0atu549thCR/gsJ1d+O</latexit><latexit sha1_base64="ouxqMrGeaeAkQ2o514v2CePE5ck=">AAACs3icbVFdSxtBFJ1sbbWxrdo+9mUxCH2S3ZZWXwpWEXxUalRIlnBn9m4yOB/LzN3QsOQn9LUF/1n/jbNJLCZ6YeBw7secew8vlfSUJP9a0Yu1l6/WN163N9+8fbe1vfP+ytvKCewKq6y74eBRSYNdkqTwpnQImiu85rcnTf56jM5Lay5pUmKmYWhkIQVQoH6OB18G251kP5lF/BSkC9Bhizgf7LTu+rkVlUZDQoH3vTQpKavBkRQKp+1+5bEEcQtD7AVoQKPP6pnWabwXmDwurAvPUDxjH3fUoL2faB4qNdDIr+Ya8rlcr6LiMKulKStCI+YfFZWKycbN4nEuHQpSkwBAOBm0xmIEDgSF8yxNEiBG+L0A5TGrtTSE+dJSNc1Uc/trmUayVs3ZYwwb4jEOpTk1Y+msaa71MKzu8ybzaM50r93/UQQlpyZ/tgFNvlJOgleN79yBm/yXGdxMV717Cq4+76cBXySdo28LXzfYR7bLPrGUHbAjdsbOWZcJNmS/2R/2N/oa9SIe5fPSqLXo+cCWItL3IC3cxw==</latexit>

v4<latexit sha1_base64="KHq1W7z4658hDNmdbtkN3cIOsHI=">AAACs3icbVFdSxtBFJ1s61esVutjX5aK0CfZLbX6IqhF2keljQrJEu7M3k0G52OZuRsMS35CX1vsP+u/6Wxii4keGDic+zHn3stLJT0lyZ9W9OLl0vLK6lp7/dXG5uut7TdX3lZOYEdYZd0NB49KGuyQJIU3pUPQXOE1v/3cxK9H6Ly05juNS8w0DIwspAAK0rdR/2N/azfZT6aIn5L0geyerH+5ZwEX/e3W715uRaXRkFDgfTdNSspqcCSFwkm7V3ksQdzCALuBGtDos3rqdRLvBSWPC+vCMxRP1ccVNWjvx5qHTA009IuxRnwu1q2oOMpqacqK0IjZR0WlYrJxM3icS4eC1DgQEE4Gr7EYggNBYT1znQSIIR4XoDxmtZaGMJ8bqqapa27v5mUka9VMPcMwIZ7hQJpzM5LOmmZb/5rVPd5EHvWZ7LV7p0Vwcm7yZwvQ5AvpJHjV3J07cOP/NsM108XbPSVXH/bTwC/DWT+xGVbZW/aOvWcpO2Qn7Cu7YB0m2ID9YD/Zr+gg6kY8ymepUeuhZofNIdJ/AaFr3fM=</latexit><latexit sha1_base64="7dzQuo6fmLYfmsedT3b8J9SGLuA=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/Krlj1pVArRR9bNG0gWcKd2bvJ0PlYZu4Gw5Kf4KtS/48/wn/jbFKlSXth4HDux5x7Dy+V9JQkf1rRk6crq2vrG+3NZ1vbz3d2X1x5WzmBHWGVdV0OHpU02CFJCrulQ9Bc4TW/+djkr8fovLTmK01KzDQMjSykAArUl/HgzWDnIDlKZhE/BOkdODjd/HS7/7ubXwx2W7/6uRWVRkNCgfe9NCkpq8GRFAqn7X7lsQRxA0PsBWhAo8/qmdZpfBiYPC6sC89QPGPvd9SgvZ9oHio10Mgv5xrysVyvouJ9VktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwpADlMau1NIT5wlI1zVRz+22RRrJWzdkzDBviGQ6lOTdj6axprvVvWN3nTebenOlhu/+hCErOTf5oA5p8qZwErxrfuQM3+S8zuJkue/cQXL0+SgO+DLa+ZfNYZ/vsJXvFUvaOnbLP7IJ1mGBD9p39YD+j46gX8Sifl0atu549thCR/gsMIt+P</latexit><latexit sha1_base64="7dzQuo6fmLYfmsedT3b8J9SGLuA=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/Krlj1pVArRR9bNG0gWcKd2bvJ0PlYZu4Gw5Kf4KtS/48/wn/jbFKlSXth4HDux5x7Dy+V9JQkf1rRk6crq2vrG+3NZ1vbz3d2X1x5WzmBHWGVdV0OHpU02CFJCrulQ9Bc4TW/+djkr8fovLTmK01KzDQMjSykAArUl/HgzWDnIDlKZhE/BOkdODjd/HS7/7ubXwx2W7/6uRWVRkNCgfe9NCkpq8GRFAqn7X7lsQRxA0PsBWhAo8/qmdZpfBiYPC6sC89QPGPvd9SgvZ9oHio10Mgv5xrysVyvouJ9VktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwpADlMau1NIT5wlI1zVRz+22RRrJWzdkzDBviGQ6lOTdj6axprvVvWN3nTebenOlhu/+hCErOTf5oA5p8qZwErxrfuQM3+S8zuJkue/cQXL0+SgO+DLa+ZfNYZ/vsJXvFUvaOnbLP7IJ1mGBD9p39YD+j46gX8Sifl0atu549thCR/gsMIt+P</latexit><latexit sha1_base64="/SWqInpbaa7y9NZAMUrCUpWhuDI=">AAACs3icbVFdSxtBFJ1sbbWxrdo+9mUxCH2S3dJWXwpWEXxUalRIlnBn9m4yOB/LzN3QsOQn9LUF/1n/jbNJLCZ6YeBw7secew8vlfSUJP9a0Yu1l6/WN163N9+8fbe1vfP+ytvKCewKq6y74eBRSYNdkqTwpnQImiu85rcnTf56jM5Lay5pUmKmYWhkIQVQoH6OB18G251kP5lF/BSkC9Bhizgf7LTu+rkVlUZDQoH3vTQpKavBkRQKp+1+5bEEcQtD7AVoQKPP6pnWabwXmDwurAvPUDxjH3fUoL2faB4qNdDIr+Ya8rlcr6LiMKulKStCI+YfFZWKycbN4nEuHQpSkwBAOBm0xmIEDgSF8yxNEiBG+L0A5TGrtTSE+dJSNc1Uc/trmUayVs3ZYwwb4jEOpTk1Y+msaa71MKzu8ybzaM50r93/UQQlpyZ/tgFNvlJOgleN79yBm/yXGdxMV717Cq4+76cBXySdo28LXzfYR7bLPrGUHbAjdsbOWZcJNmS/2R/2N/oa9SIe5fPSqLXo+cCWItL3InrcyA==</latexit>

v5<latexit sha1_base64="PmhvU5TTpwFAMMF4IJlMP0j75aM=">AAACs3icbVFdSxtBFJ2sbbWxrVof+7IoQp9kt+DHi2At0j5aNCokS7gzezcZnI9l5m4wLPkJfbW0/8x/09nEiokeGDic+zHn3stLJT0lyX0rWnr1+s3yytv26rv3H9bWNz5eels5gR1hlXXXHDwqabBDkhRelw5Bc4VX/OZbE78aofPSmgsal5hpGBhZSAEUpPNRf6+/vp3sJlPEz0n6QLaPV7//YQFn/Y3W315uRaXRkFDgfTdNSspqcCSFwkm7V3ksQdzAALuBGtDos3rqdRLvBCWPC+vCMxRP1acVNWjvx5qHTA009IuxRnwp1q2oOMxqacqK0IjZR0WlYrJxM3icS4eC1DgQEE4Gr7EYggNBYT1znQSIIR4VoDxmtZaGMJ8bqqapa25v52Uka9VMPcEwIZ7gQJpTM5LOmmZb/5vVPd5EnvSZ7LR7X4vg5NTkLxagyRfSSfCquTt34MaPNsM108XbPSeXX3bTwH+Gs+6zGVbYJ7bFPrOUHbBj9oOdsQ4TbMB+sTv2O9qLuhGP8llq1Hqo2WRziPQ/o7jd9A==</latexit><latexit sha1_base64="ynrnXb0FlT3gZE68UsMHKgA8Sbw=">AAACs3icbVFbSxtBFJ5sL2rsRdtHXxZF6JPsFrR9KahF7KOlRgPJEs7Mnk0G57LMnA2GJT/B15b6f/oj+m+cTWwx0QMDH9+5zHfOx0slPSXJ31b07PmLlyura+31V6/fvN3YfHfhbeUEdoRV1nU5eFTSYIckKeyWDkFzhZf86muTvxyj89Kac5qUmGkYGllIARSoH+PB/mBjJ9lLZhE/Buk92DlcP/299aebnw02W7f93IpKoyGhwPtempSU1eBICoXTdr/yWIK4giH2AjSg0Wf1TOs03g1MHhfWhWconrEPO2rQ3k80D5UaaOSXcw35VK5XUfE5q6UpK0Ij5h8VlYrJxs3icS4dClKTAEA4GbTGYgQOBIXzLEwSIEb4pQDlMau1NIT5wlI1zVRze71II1mr5uwxhg3xGIfSnJixdNY01/o3rO7zJvNgznS33T8qgpITkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GFx/30oC/B1sP2DxW2RbbZh9Yyj6xQ/aNnbEOE2zIbthP9ivaj3oRj/J5adS673nPFiLSdw5v35A=</latexit><latexit sha1_base64="ynrnXb0FlT3gZE68UsMHKgA8Sbw=">AAACs3icbVFbSxtBFJ5sL2rsRdtHXxZF6JPsFrR9KahF7KOlRgPJEs7Mnk0G57LMnA2GJT/B15b6f/oj+m+cTWwx0QMDH9+5zHfOx0slPSXJ31b07PmLlyura+31V6/fvN3YfHfhbeUEdoRV1nU5eFTSYIckKeyWDkFzhZf86muTvxyj89Kac5qUmGkYGllIARSoH+PB/mBjJ9lLZhE/Buk92DlcP/299aebnw02W7f93IpKoyGhwPtempSU1eBICoXTdr/yWIK4giH2AjSg0Wf1TOs03g1MHhfWhWconrEPO2rQ3k80D5UaaOSXcw35VK5XUfE5q6UpK0Ij5h8VlYrJxs3icS4dClKTAEA4GbTGYgQOBIXzLEwSIEb4pQDlMau1NIT5wlI1zVRze71II1mr5uwxhg3xGIfSnJixdNY01/o3rO7zJvNgznS33T8qgpITkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GFx/30oC/B1sP2DxW2RbbZh9Yyj6xQ/aNnbEOE2zIbthP9ivaj3oRj/J5adS673nPFiLSdw5v35A=</latexit><latexit sha1_base64="dNLHeHZxGS+lO1dEVWFO8fhHNc0=">AAACs3icbVFdSxtBFJ1sbWtjW7U+9mUxCD7JbsHWl4JaBB+VNiokS7gzezcZnI9l5m4wLPkJfW3Bf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrerX2+s3b9XftjfcfPm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNb9oUmKmYWhkIQVQoH6OB4eDrU5ykMwifg7SBeiwRVwMtlv3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4LTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAj/F6A8pjVWhrCfGmpmmaqub1bppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme61+6fFEHJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6svB2nAl0nn+OvC13X2me2yfZayb+yYnbML1mWCDdlv9of9jQ6jXsSjfF4atRY9O2wpIv0IJMfcyQ==</latexit>

v6<latexit sha1_base64="wKhrxldwWBsyuZ/SMWKlYGYwcYE=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLVT7UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9Df2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVX7JamrIiNGL2UVGpmGzcDB7n0qEgNQ4EhJPBayyG4EBQWM9cJwFiiF8LUB6zWktDmM8NVdPUNbc38zKStWqmnmCYEE9wIM2pGUlnTbOtf83qHm8iT/pMdtu94yI4OTX5iwVo8oV0Erxq7s4duPF/m+Ga6eLtnpOLT/tp4OfhrAdshhX2kW2zPZayQ3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0XpgXd9Q==</latexit><latexit sha1_base64="eF8kon3iiIVAatz0rMVtj7XtVjI=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PBwWBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxC835E=</latexit><latexit sha1_base64="eF8kon3iiIVAatz0rMVtj7XtVjI=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PBwWBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxC835E=</latexit><latexit sha1_base64="UkfF8c+IMd3jOYhFfwj6/krSo64=">AAACs3icbVFdSxtBFJ1sa7VpbbV97MtiEPoku4LWF8FahD5aalRIlnBn9m4yOB/LzN1gWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebHz5ubX+69LZyArvCKuuuOXhU0mCXJCm8Lh2C5gqv+M2PJn81RuelNRc0KTHTMDSykAIoUL/Hg8PBVifZS2YRPwfpAnTYIs4H2637fm5FpdGQUOB9L01KympwJIXCabtfeSxB3MAQewEa0OizeqZ1Gu8GJo8L68IzFM/Ypx01aO8nmodKDTTyq7mGfCnXq6g4ymppyorQiPlHRaVisnGzeJxLh4LUJAAQTgatsRiBA0HhPEuTBIgRHhegPGa1loYwX1qqpplqbm+XaSRr1Zw9xbAhnuJQmjMzls6a5lr/htV93mSezJnutvvfi6DkzOQvNqDJV8pJ8KrxnTtwk/8yg5vpqnfPweX+Xhrwr6RzcrjwdYN9YTvsK0vZN3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWouez2wpIv0IJxTcyg==</latexit>

v7<latexit sha1_base64="dB/GJtb3D49gFl2WcODK2B4yvg8=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLVT7UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9jf2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVX7JamrIiNGL2UVGpmGzcDB7n0qEgNQ4EhJPBayyG4EBQWM9cJwFiiF8LUB6zWktDmM8NVdPUNbc38zKStWqmnmCYEE9wIM2pGUlnTbOtf83qHm8iT/pMdtu94yI4OTX5iwVo8oV0Erxq7s4duPF/m+Ga6eLtnpOLT/tp4OfhrAdshhX2kW2zPZayQ3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0XqFLd9g==</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit>

v8<latexit sha1_base64="FeDIxN3se0wzbthPl/n39Dpz7Eo=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLdT6UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9jf2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVh1ktTVkRGjH7qKhUTDZuBo9z6VCQGgcCwsngNRZDcCAorGeukwAxxK8FKI9ZraUhzOeGqmnqmtubeRnJWjVTTzBMiCc4kObUjKSzptnWv2Z1jzeRJ30mu+3ecRGcnJr8xQI0+UI6CV41d+cO3Pi/zXDNdPF2z8nFp/008PNw1gM2wwr7yLbZHkvZF3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0Xqp/d9w==</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="xunXA1IUYOSicXgGkapNqZJQmpY=">AAACs3icbVFdSxtBFJ1sa2vTWrV99GUxCH2SXcHWF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNZc0KTHTMDSykAIoUL/Gg6PBVifZT2YRPwfpAnTYIs4H2637fm5FpdGQUOB9L01KympwJIXCabtfeSxB3MIQewEa0OizeqZ1Gu8FJo8L68IzFM/Ypx01aO8nmodKDTTyq7mGfCnXq6g4ymppyorQiPlHRaVisnGzeJxLh4LUJAAQTgatsRiBA0HhPEuTBIgRHhegPGa1loYwX1qqpplqbn8v00jWqjl7imFDPMWhNGdmLJ01zbX+Dav7vMk8mTPda/e/F0HJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6uD/TTgi6Rz8nXh6zrbYbvsC0vZN3bCfrJz1mWCDdkf9pfdRYdRL+JRPi+NWouez2wpIv0IK67czA==</latexit>

v9<latexit sha1_base64="+2w3CQFWqo7Ry7tSEVHmpN9MmAE=">AAACs3icbVFdSxtBFJ1s61esVutjX5aK0CfZLdTqg6AWaR+VNiokS7gzezcZnI9l5m4wLPkJfW2x/6z/prOJLSZ6YOBw7secey8vlfSUJH9a0YuXS8srq2vt9Vcbm6+3tt9ceVs5gR1hlXU3HDwqabBDkhTelA5Bc4XX/PZzE78eofPSmu80LjHTMDCykAIoSN9G/aP+1m6yn0wRPyXpA9k9Wf9yzwIu+tut373cikqjIaHA+26alJTV4EgKhZN2r/JYgriFAXYDNaDRZ/XU6yTeC0oeF9aFZyieqo8ratDejzUPmRpo6BdjjfhcrFtRcZjV0pQVoRGzj4pKxWTjZvA4lw4FqXEgIJwMXmMxBAeCwnrmOgkQQzwuQHnMai0NYT43VE1T19zezctI1qqZeoZhQjzDgTTnZiSdNc22/jWre7yJPOoz2Wv3Tovg5NzkzxagyRfSSfCquTt34Mb/bYZrpou3e0quPuyngV+Gsx6wGVbZW/aOvWcp+8RO2Fd2wTpMsAH7wX6yX9HHqBvxKJ+lRq2Hmh02h0j/Bazs3fg=</latexit><latexit sha1_base64="sO52qWhbLJdL9GPtCTBtMMv0KKY=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/KrmDVh0KtFH1s0bSBZAl3Zu8mQ+djmbkbDEt+gq9K/T/+CP+Ns0mVJu2FgcO5H3PuPbxU0lOS/GlFT56urK6tb7Q3n21tP9/ZfXHlbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85mOTvx6j89KarzQpMdMwNLKQAihQX8aD94Odg+QomUX8EKR34OB089Pt/u9ufjHYbf3q51ZUGg0JBd730qSkrAZHUiictvuVxxLEDQyxF6ABjT6rZ1qn8WFg8riwLjxD8Yy931GD9n6ieajUQCO/nGvIx3K9iop3WS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2GSADHCkwKUx6zW0hDmC0vVNFPN7bdFGslaNWfPMGyIZziU5tyMpbOmuda/YXWfN5l7c6aH7f6HIig5N/mjDWjypXISvGp85w7c5L/M4Ga67N1DcPX6KA34Mth6zOaxzvbZS/aKpewtO2Wf2QXrMMGG7Dv7wX5Gb6JexKN8Xhq17nr22EJE+i8Xo9+U</latexit><latexit sha1_base64="sO52qWhbLJdL9GPtCTBtMMv0KKY=">AAACs3icbVFdaxNBFJ2s9iu1tbWPfVksBZ/KrmDVh0KtFH1s0bSBZAl3Zu8mQ+djmbkbDEt+gq9K/T/+CP+Ns0mVJu2FgcO5H3PuPbxU0lOS/GlFT56urK6tb7Q3n21tP9/ZfXHlbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85mOTvx6j89KarzQpMdMwNLKQAihQX8aD94Odg+QomUX8EKR34OB089Pt/u9ufjHYbf3q51ZUGg0JBd730qSkrAZHUiictvuVxxLEDQyxF6ABjT6rZ1qn8WFg8riwLjxD8Yy931GD9n6ieajUQCO/nGvIx3K9iop3WS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2GSADHCkwKUx6zW0hDmC0vVNFPN7bdFGslaNWfPMGyIZziU5tyMpbOmuda/YXWfN5l7c6aH7f6HIig5N/mjDWjypXISvGp85w7c5L/M4Ga67N1DcPX6KA34Mth6zOaxzvbZS/aKpewtO2Wf2QXrMMGG7Dv7wX5Gb6JexKN8Xhq17nr22EJE+i8Xo9+U</latexit><latexit sha1_base64="pXXcvq7sBw1aB7hDEnOZG1AKBYg=">AAACs3icbVFdSxtBFJ1sbbWxrdo+9mUxCH2S3UJbfShYRfBRqVEhWcKd2bvJ4HwsM3dDw5Kf0NcW/Gf9N84msZjohYHDuR9z7j28VNJTkvxrRS/WXr5a33jd3nzz9t3W9s77K28rJ7ArrLLuhoNHJQ12SZLCm9IhaK7wmt+eNPnrMTovrbmkSYmZhqGRhRRAgfo5HhwOtjvJfjKL+ClIF6DDFnE+2Gnd9XMrKo2GhALve2lSUlaDIykUTtv9ymMJ4haG2AvQgEaf1TOt03gvMHlcWBeeoXjGPu6oQXs/0TxUaqCRX8015HO5XkXFQVZLU1aERsw/KioVk42bxeNcOhSkJgGAcDJojcUIHAgK51maJECM8HsBymNWa2kI86Wlapqp5vbXMo1krZqzxxg2xGMcSnNqxtJZ01zrYVjd503m0ZzpXrv/owhKTk3+bAOafKWcBK8a37kDN/kvM7iZrnr3FFx93k8Dvkg6R18Xvm6wj2yXfWIp+8aO2Bk7Z10m2JD9Zn/Y3+hL1It4lM9Lo9ai5wNbikjfAy373M0=</latexit>

2666666666666664

⇥ ⇥ ⇥ ⇥⇥ ⇥ ⇥ ⇥ ⇥ ⇥ ⇥

⇥ ⇥⇥ ⇥

⇥ ⇥ ⇥⇥

⇥ ⇥⇥ ⇥ ⇥

⇥ ⇥ ⇥ ⇥⇥ ⇥ ⇥

3777777777777775

<latexit sha1_base64="xDGNmPi5UCwRmf2ZkaSsiSTZmGo=">AAAFmXichZJbTxNBFMeX1SquF0AfeRnsJcYYsgsI+mDCRQ0xMSDIJek0ZHZ62k6Y2d3MTBGy2Rc/ja/6bfw2ni1LZReLk5z23//5nTOXnjCRwljf/z3l3rlbu3d/+oH38NHjJzOzc08PTTzUHA54LGN9HDIDUkRwYIWVcJxoYCqUcBSebuX5ozPQRsTRV3uRQEexfiR6gjOL1smcO99oeFRCz7Y9GkJfRCnTml1kKR+vzGsSn7RIgLFE8GMZYwXjNcYqxhrGG4y3hFDqtegCtUKBoQut1uQffzWlTd/7d6qk/29f71k5R/UkmG9V7euSNpe9CXtX65BdubHb1R5Vu7THrTuUyUm3n/xCk7tXHwZJiLrFH+9RLfoD2/EajZPZur/ojxa5KYJC1J1i7eI4Ae3GfKggslwyY9qBn9gOdraCS8DeQwMJ46esD22UEcNTdNLRGGekiU6X9GKNEVkycq9XpEwZc6FCJBWzA1PN5eY4V0rm/UwCOMm0Cz2KRD1I8c5mqCHH09wahL20HmRZAZ0Bvx2K4BuPlWL4cnQ/4Vk76BRIOELKxE6iCyKMz9NGSjmTBLFGFdwDJjNCyLjXXpXYilUi4TwbE1tZNum6BqxiIsqddEMLbF14dqBV1TG9qmNLVWEsu5eV7U2UH7Hpu3QnwVli5MO51WxT4gmyzpWXQ1j5HnAWNHzGyp0ENLOxfplSpvuK4SXwm77K1W2giK5AVJ6HgxlUx/CmOFxaDFB/WaqvrxYjOu3MO8+dF07grDnrzraz6xw43P3u/nB/ur9q87WN2nbt0yXqThU1z5zSqu3/AXbOxLQ=</latexit><latexit sha1_base64="xDGNmPi5UCwRmf2ZkaSsiSTZmGo=">AAAFmXichZJbTxNBFMeX1SquF0AfeRnsJcYYsgsI+mDCRQ0xMSDIJek0ZHZ62k6Y2d3MTBGy2Rc/ja/6bfw2ni1LZReLk5z23//5nTOXnjCRwljf/z3l3rlbu3d/+oH38NHjJzOzc08PTTzUHA54LGN9HDIDUkRwYIWVcJxoYCqUcBSebuX5ozPQRsTRV3uRQEexfiR6gjOL1smcO99oeFRCz7Y9GkJfRCnTml1kKR+vzGsSn7RIgLFE8GMZYwXjNcYqxhrGG4y3hFDqtegCtUKBoQut1uQffzWlTd/7d6qk/29f71k5R/UkmG9V7euSNpe9CXtX65BdubHb1R5Vu7THrTuUyUm3n/xCk7tXHwZJiLrFH+9RLfoD2/EajZPZur/ojxa5KYJC1J1i7eI4Ae3GfKggslwyY9qBn9gOdraCS8DeQwMJ46esD22UEcNTdNLRGGekiU6X9GKNEVkycq9XpEwZc6FCJBWzA1PN5eY4V0rm/UwCOMm0Cz2KRD1I8c5mqCHH09wahL20HmRZAZ0Bvx2K4BuPlWL4cnQ/4Vk76BRIOELKxE6iCyKMz9NGSjmTBLFGFdwDJjNCyLjXXpXYilUi4TwbE1tZNum6BqxiIsqddEMLbF14dqBV1TG9qmNLVWEsu5eV7U2UH7Hpu3QnwVli5MO51WxT4gmyzpWXQ1j5HnAWNHzGyp0ENLOxfplSpvuK4SXwm77K1W2giK5AVJ6HgxlUx/CmOFxaDFB/WaqvrxYjOu3MO8+dF07grDnrzraz6xw43P3u/nB/ur9q87WN2nbt0yXqThU1z5zSqu3/AXbOxLQ=</latexit><latexit sha1_base64="xDGNmPi5UCwRmf2ZkaSsiSTZmGo=">AAAFmXichZJbTxNBFMeX1SquF0AfeRnsJcYYsgsI+mDCRQ0xMSDIJek0ZHZ62k6Y2d3MTBGy2Rc/ja/6bfw2ni1LZReLk5z23//5nTOXnjCRwljf/z3l3rlbu3d/+oH38NHjJzOzc08PTTzUHA54LGN9HDIDUkRwYIWVcJxoYCqUcBSebuX5ozPQRsTRV3uRQEexfiR6gjOL1smcO99oeFRCz7Y9GkJfRCnTml1kKR+vzGsSn7RIgLFE8GMZYwXjNcYqxhrGG4y3hFDqtegCtUKBoQut1uQffzWlTd/7d6qk/29f71k5R/UkmG9V7euSNpe9CXtX65BdubHb1R5Vu7THrTuUyUm3n/xCk7tXHwZJiLrFH+9RLfoD2/EajZPZur/ojxa5KYJC1J1i7eI4Ae3GfKggslwyY9qBn9gOdraCS8DeQwMJ46esD22UEcNTdNLRGGekiU6X9GKNEVkycq9XpEwZc6FCJBWzA1PN5eY4V0rm/UwCOMm0Cz2KRD1I8c5mqCHH09wahL20HmRZAZ0Bvx2K4BuPlWL4cnQ/4Vk76BRIOELKxE6iCyKMz9NGSjmTBLFGFdwDJjNCyLjXXpXYilUi4TwbE1tZNum6BqxiIsqddEMLbF14dqBV1TG9qmNLVWEsu5eV7U2UH7Hpu3QnwVli5MO51WxT4gmyzpWXQ1j5HnAWNHzGyp0ENLOxfplSpvuK4SXwm77K1W2giK5AVJ6HgxlUx/CmOFxaDFB/WaqvrxYjOu3MO8+dF07grDnrzraz6xw43P3u/nB/ur9q87WN2nbt0yXqThU1z5zSqu3/AXbOxLQ=</latexit><latexit sha1_base64="xDGNmPi5UCwRmf2ZkaSsiSTZmGo=">AAAFmXichZJbTxNBFMeX1SquF0AfeRnsJcYYsgsI+mDCRQ0xMSDIJek0ZHZ62k6Y2d3MTBGy2Rc/ja/6bfw2ni1LZReLk5z23//5nTOXnjCRwljf/z3l3rlbu3d/+oH38NHjJzOzc08PTTzUHA54LGN9HDIDUkRwYIWVcJxoYCqUcBSebuX5ozPQRsTRV3uRQEexfiR6gjOL1smcO99oeFRCz7Y9GkJfRCnTml1kKR+vzGsSn7RIgLFE8GMZYwXjNcYqxhrGG4y3hFDqtegCtUKBoQut1uQffzWlTd/7d6qk/29f71k5R/UkmG9V7euSNpe9CXtX65BdubHb1R5Vu7THrTuUyUm3n/xCk7tXHwZJiLrFH+9RLfoD2/EajZPZur/ojxa5KYJC1J1i7eI4Ae3GfKggslwyY9qBn9gOdraCS8DeQwMJ46esD22UEcNTdNLRGGekiU6X9GKNEVkycq9XpEwZc6FCJBWzA1PN5eY4V0rm/UwCOMm0Cz2KRD1I8c5mqCHH09wahL20HmRZAZ0Bvx2K4BuPlWL4cnQ/4Vk76BRIOELKxE6iCyKMz9NGSjmTBLFGFdwDJjNCyLjXXpXYilUi4TwbE1tZNum6BqxiIsqddEMLbF14dqBV1TG9qmNLVWEsu5eV7U2UH7Hpu3QnwVli5MO51WxT4gmyzpWXQ1j5HnAWNHzGyp0ENLOxfplSpvuK4SXwm77K1W2giK5AVJ6HgxlUx/CmOFxaDFB/WaqvrxYjOu3MO8+dF07grDnrzraz6xw43P3u/nB/ur9q87WN2nbt0yXqThU1z5zSqu3/AXbOxLQ=</latexit>

v1<latexit sha1_base64="L/0JIBe9OYFCCCQ2Ju6OP8PDaKI=">AAACs3icbVHbahRBEK0db3HjJdFHXwZDwKcwI3h5EWIk6GNENwnsDkt1T81uk74M3TWLy7Cf4Kuif+bf2LMbJbvJgYbDqUufqhK1VoGz7E8vuXX7zt17W/f72w8ePnq8s/vkNLjGSxpIp50/FxhIK0sDVqzpvPaERmg6ExcfuvjZjHxQzn7leU2FwYlVlZLIUfoyG+fjnb3sIFsivU7yS7J3uP3xF0ScjHd7v0elk40hy1JjCMM8q7lo0bOSmhb9UROoRnmBExpGatFQKNql10W6H5UyrZyPz3K6VK9WtGhCmBsRMw3yNGzGOvGm2LDh6m3RKls3TFauPqoanbJLu8HTUnmSrOeRoPQqek3lFD1KjutZ6yRRTuldhTpQ0Rplmcq1oVpeuhbu27pM7JxeqUcUJ6Qjmih7bGfKO9tt61+zdiS6yJU+i/3+6H0VnRzb8sYCsuVGOkvRdHcXHv38v814zXzzdtfJ6cuDPPLP8ayvYYUteAbP4QXk8AYO4ROcwAAkTOA7/ICfyatkmIikXKUmvcuap7CGxPwFmoTd8A==</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="PPwaTwye9fNx5nrXxNntNOXTfm0=">AAAConicbVFNT9tAEN24paQBCpx7sYgi9YRsLnBBAiqkHlOJkEiJC7PrcbJivWvtjlEjK3+Aa2/8M/5N10lAJGSklZ7efOybebxQ0lEUvTSCT5+3vmw3v7Z2dlt73/YPdm+dKa3AnjDK2AEHh0pq7JEkhYPCIuRcYZ8//Kzz/Ue0Thp9Q9MCkxzGWmZSAHmqe3fQjo6jeYQfQbwEbbaMu8PG8yg1osxRk1Dg3DCOCkoqsCSFwllrVDosQDzAGIceasjRJdVc5yzseCYNM2P90xTO2fcdFeTOTXPuK3OgiVvP1eSm3LCk7CyppC5KQi0WH2WlCsmE9dJhKi0KUlMPQFjptYZiAhYE+dOsTBIgJniegXKYVLnUhOnKUhXNVXPzd5VGMkYt2Cv0G+IVjqW+1o/SGl1f63VYNeJ15t2cWac1usy8kmudbmxAna6Vk+Bl7Tm3YKdvMr2Z8bp1H8HtyXHs8e+INdl3dsR+sJidsgv2i3VZjwmWsif2L4iDfvBnYXrQWLp/yFYiuP8PK+bZpQ==</latexit><latexit sha1_base64="UUgNIQug54JXhWWfihFsCBCpYec=">AAACqHicbVJRSxtBEJ5cbbWprdrXvhwVwSe5K5T6IrQWwUeLjQrJEWb35pLFvd1jdy4YjvyEvrbgP+u/6V4Si4kOLHx83+zsNzMrKq08J8nfTvRi4+Wrza3X3Tfbb9/t7O5tX3lbO0k9abV1NwI9aWWox4o13VSOsBSarsXt91a/npDzypqfPK0oK3FkVKEkcqAuJ8N0uLufHCXziJ+CdAn2YRkXw73O/SC3si7JsNTofT9NKs4adKykpll3UHuqUN7iiPoBGizJZ83c6yw+CEweF9aFYzies49vNFh6Py1FyCyRx35da8nntH7NxXHWKFPVTEYuHipqHbON28bjXDmSrKcBoHQqeI3lGB1KDuNZqSRRjumkQO0pa0plmPKVphqeuxb2bpUmtlYv2FMKHdIpjZQ5MxPlrGmn9VCsGYhWeVRndtAdfCuCkzOTP3uBTL6WzlLU7d6FQzf9bzNsM13f3VNw9ekoDfhHAlvwAT7CIaTwBb7COVxADySM4Bf8hj/R56gficXeo87yA7yHlYjyf5V/27Y=</latexit><latexit sha1_base64="UUgNIQug54JXhWWfihFsCBCpYec=">AAACqHicbVJRSxtBEJ5cbbWprdrXvhwVwSe5K5T6IrQWwUeLjQrJEWb35pLFvd1jdy4YjvyEvrbgP+u/6V4Si4kOLHx83+zsNzMrKq08J8nfTvRi4+Wrza3X3Tfbb9/t7O5tX3lbO0k9abV1NwI9aWWox4o13VSOsBSarsXt91a/npDzypqfPK0oK3FkVKEkcqAuJ8N0uLufHCXziJ+CdAn2YRkXw73O/SC3si7JsNTofT9NKs4adKykpll3UHuqUN7iiPoBGizJZ83c6yw+CEweF9aFYzies49vNFh6Py1FyCyRx35da8nntH7NxXHWKFPVTEYuHipqHbON28bjXDmSrKcBoHQqeI3lGB1KDuNZqSRRjumkQO0pa0plmPKVphqeuxb2bpUmtlYv2FMKHdIpjZQ5MxPlrGmn9VCsGYhWeVRndtAdfCuCkzOTP3uBTL6WzlLU7d6FQzf9bzNsM13f3VNw9ekoDfhHAlvwAT7CIaTwBb7COVxADySM4Bf8hj/R56gficXeo87yA7yHlYjyf5V/27Y=</latexit><latexit sha1_base64="lYAi3ezFm8u7WvA2S2wLievrc8A=">AAACs3icbVFdSxtBFJ1sa7Vprdo+9mUxCH2S3UK1LwWrCH201KiQLOHO7N1kcD6WmbvBsOQn9FXBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebH7a2dz5eeFs5gV1hlXVXHDwqabBLkhRelQ5Bc4WX/PqkyV+O0XlpzTlNSsw0DI0spAAK1J/xIB1sd5L9ZBbxc5AuQIct4myw07rv51ZUGg0JBd730qSkrAZHUiictvuVxxLENQyxF6ABjT6rZ1qn8V5g8riwLjxD8Yx92lGD9n6ieajUQCO/mmvIl3K9iorvWS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2mSADHCHwUoj1mtpSHMl5aqaaaa25tlGslaNWePMWyIxziU5tSMpbOmuda/YXWfN5knc6Z77f7PIig5NfmLDWjylXISvGp85w7c5L/M4Ga66t1zcPF1Pw34d9I5Olj4usE+s132haXskB2xX+yMdZlgQ/aX3bK76FvUi3iUz0uj1qLnE1uKSD8CG5PcxQ==</latexit><latexit sha1_base64="lYAi3ezFm8u7WvA2S2wLievrc8A=">AAACs3icbVFdSxtBFJ1sa7Vprdo+9mUxCH2S3UK1LwWrCH201KiQLOHO7N1kcD6WmbvBsOQn9FXBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebH7a2dz5eeFs5gV1hlXVXHDwqabBLkhRelQ5Bc4WX/PqkyV+O0XlpzTlNSsw0DI0spAAK1J/xIB1sd5L9ZBbxc5AuQIct4myw07rv51ZUGg0JBd730qSkrAZHUiictvuVxxLENQyxF6ABjT6rZ1qn8V5g8riwLjxD8Yx92lGD9n6ieajUQCO/mmvIl3K9iorvWS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2mSADHCHwUoj1mtpSHMl5aqaaaa25tlGslaNWePMWyIxziU5tSMpbOmuda/YXWfN5knc6Z77f7PIig5NfmLDWjylXISvGp85w7c5L/M4Ga66t1zcPF1Pw34d9I5Olj4usE+s132haXskB2xX+yMdZlgQ/aX3bK76FvUi3iUz0uj1qLnE1uKSD8CG5PcxQ==</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="/mAjvKGWemnMSsqnrP2CESae5J0=">AAACs3icbVFdSxtBFJ1sbWujrdo++rIogk+yW2jrS8FapH201GggWcKd2bvJ4HwsM3dDw5Kf0FdF/09/RP9NZxMrJnph4HDux5x7Dy+V9JQkf1vRs5XnL16uvmqvrb9+s7G59fbc28oJ7AirrOty8KikwQ5JUtgtHYLmCi/45dcmfzFG56U1ZzQpMdMwNLKQAihQP8eDdLC5mxwks4gfg/QO7B6tfbvZ/tPNTwdbrdt+bkWl0ZBQ4H0vTUrKanAkhcJpu195LEFcwhB7ARrQ6LN6pnUa7wUmjwvrwjMUz9iHHTVo7yeah0oNNPLLuYZ8KterqDjMamnKitCI+UdFpWKycbN4nEuHgtQkABBOBq2xGIEDQeE8C5MEiBF+LkB5zGotDWG+sFRNM9Xc/lqkkaxVc/YYw4Z4jENpTsxYOmuaa/0fVvd5k3kwZ7rX7n8pgpITkz/ZgCZfKifBq8Z37sBN7mUGN9Nl7x6D8/cHacA/gq0f2TxW2TbbYfssZZ/YEfvOTlmHCTZkv9kVu44+RL2IR/m8NGrd9bxjCxHpfwU734w=</latexit><latexit sha1_base64="lYAi3ezFm8u7WvA2S2wLievrc8A=">AAACs3icbVFdSxtBFJ1sa7Vprdo+9mUxCH2S3UK1LwWrCH201KiQLOHO7N1kcD6WmbvBsOQn9FXBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c36xtv2u/ebH7a2dz5eeFs5gV1hlXVXHDwqabBLkhRelQ5Bc4WX/PqkyV+O0XlpzTlNSsw0DI0spAAK1J/xIB1sd5L9ZBbxc5AuQIct4myw07rv51ZUGg0JBd730qSkrAZHUiictvuVxxLENQyxF6ABjT6rZ1qn8V5g8riwLjxD8Yx92lGD9n6ieajUQCO/mmvIl3K9iorvWS1NWREaMf+oqFRMNm4Wj3PpUJCaBADCyaA1FiNwICicZ2mSADHCHwUoj1mtpSHMl5aqaaaa25tlGslaNWePMWyIxziU5tSMpbOmuda/YXWfN5knc6Z77f7PIig5NfmLDWjylXISvGp85w7c5L/M4Ga66t1zcPF1Pw34d9I5Olj4usE+s132haXskB2xX+yMdZlgQ/aX3bK76FvUi3iUz0uj1qLnE1uKSD8CG5PcxQ==</latexit>

v7<latexit sha1_base64="dB/GJtb3D49gFl2WcODK2B4yvg8=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLVT7UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9jf2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVX7JamrIiNGL2UVGpmGzcDB7n0qEgNQ4EhJPBayyG4EBQWM9cJwFiiF8LUB6zWktDmM8NVdPUNbc38zKStWqmnmCYEE9wIM2pGUlnTbOtf83qHm8iT/pMdtu94yI4OTX5iwVo8oV0Erxq7s4duPF/m+Ga6eLtnpOLT/tp4OfhrAdshhX2kW2zPZayQ3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0XqFLd9g==</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="PPwaTwye9fNx5nrXxNntNOXTfm0=">AAAConicbVFNT9tAEN24paQBCpx7sYgi9YRsLnBBAiqkHlOJkEiJC7PrcbJivWvtjlEjK3+Aa2/8M/5N10lAJGSklZ7efOybebxQ0lEUvTSCT5+3vmw3v7Z2dlt73/YPdm+dKa3AnjDK2AEHh0pq7JEkhYPCIuRcYZ8//Kzz/Ue0Thp9Q9MCkxzGWmZSAHmqe3fQjo6jeYQfQbwEbbaMu8PG8yg1osxRk1Dg3DCOCkoqsCSFwllrVDosQDzAGIceasjRJdVc5yzseCYNM2P90xTO2fcdFeTOTXPuK3OgiVvP1eSm3LCk7CyppC5KQi0WH2WlCsmE9dJhKi0KUlMPQFjptYZiAhYE+dOsTBIgJniegXKYVLnUhOnKUhXNVXPzd5VGMkYt2Cv0G+IVjqW+1o/SGl1f63VYNeJ15t2cWac1usy8kmudbmxAna6Vk+Bl7Tm3YKdvMr2Z8bp1H8HtyXHs8e+INdl3dsR+sJidsgv2i3VZjwmWsif2L4iDfvBnYXrQWLp/yFYiuP8PK+bZpQ==</latexit><latexit sha1_base64="54esQxMz4DTDx/N/CM7J5qmXIdY=">AAACqHicbVJRSxtBEJ5ctbVRW+1rXw5F8EnuBLEvhVYR+mhpo0JyhNm9uWRxb/fYnQuGIz+hry30n/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTvRiY/Plq63X3e2d3Tdv9/Z3brytnaSetNq6O4GetDLUY8Wa7ipHWApNt+L+stVvJ+S8suY7TyvKShwZVSiJHKhvk+H5cO8wOUnmET8F6RIcwjKuh/ud34Pcyrokw1Kj9/00qThr0LGSmmbdQe2pQnmPI+oHaLAknzVzr7P4KDB5XFgXjuF4zj6+0WDp/bQUIbNEHvt1rSWf0/o1Fx+yRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckwfC9SesqZUhilfaarhuWthH1ZpYmv1gr2g0CFd0EiZKzNRzpp2Wv+KNQPRKo/qzI66g89FcHJl8mcvkMnX0lmKut27cOim/22Gbabru3sKbk5P0oC/JrAF7+EAjiGFc/gEX+AaeiBhBD/gJ/yKzqJ+JBZ7jzrLD/AOViLK/wKjC9u8</latexit><latexit sha1_base64="54esQxMz4DTDx/N/CM7J5qmXIdY=">AAACqHicbVJRSxtBEJ5ctbVRW+1rXw5F8EnuBLEvhVYR+mhpo0JyhNm9uWRxb/fYnQuGIz+hry30n/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTvRiY/Plq63X3e2d3Tdv9/Z3brytnaSetNq6O4GetDLUY8Wa7ipHWApNt+L+stVvJ+S8suY7TyvKShwZVSiJHKhvk+H5cO8wOUnmET8F6RIcwjKuh/ud34Pcyrokw1Kj9/00qThr0LGSmmbdQe2pQnmPI+oHaLAknzVzr7P4KDB5XFgXjuF4zj6+0WDp/bQUIbNEHvt1rSWf0/o1Fx+yRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckwfC9SesqZUhilfaarhuWthH1ZpYmv1gr2g0CFd0EiZKzNRzpp2Wv+KNQPRKo/qzI66g89FcHJl8mcvkMnX0lmKut27cOim/22Gbabru3sKbk5P0oC/JrAF7+EAjiGFc/gEX+AaeiBhBD/gJ/yKzqJ+JBZ7jzrLD/AOViLK/wKjC9u8</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit>

v8<latexit sha1_base64="FeDIxN3se0wzbthPl/n39Dpz7Eo=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLdT6UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9jf2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVh1ktTVkRGjH7qKhUTDZuBo9z6VCQGgcCwsngNRZDcCAorGeukwAxxK8FKI9ZraUhzOeGqmnqmtubeRnJWjVTTzBMiCc4kObUjKSzptnWv2Z1jzeRJ30mu+3ecRGcnJr8xQI0+UI6CV41d+cO3Pi/zXDNdPF2z8nFp/008PNw1gM2wwr7yLbZHkvZF3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0Xqp/d9w==</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="PPwaTwye9fNx5nrXxNntNOXTfm0=">AAAConicbVFNT9tAEN24paQBCpx7sYgi9YRsLnBBAiqkHlOJkEiJC7PrcbJivWvtjlEjK3+Aa2/8M/5N10lAJGSklZ7efOybebxQ0lEUvTSCT5+3vmw3v7Z2dlt73/YPdm+dKa3AnjDK2AEHh0pq7JEkhYPCIuRcYZ8//Kzz/Ue0Thp9Q9MCkxzGWmZSAHmqe3fQjo6jeYQfQbwEbbaMu8PG8yg1osxRk1Dg3DCOCkoqsCSFwllrVDosQDzAGIceasjRJdVc5yzseCYNM2P90xTO2fcdFeTOTXPuK3OgiVvP1eSm3LCk7CyppC5KQi0WH2WlCsmE9dJhKi0KUlMPQFjptYZiAhYE+dOsTBIgJniegXKYVLnUhOnKUhXNVXPzd5VGMkYt2Cv0G+IVjqW+1o/SGl1f63VYNeJ15t2cWac1usy8kmudbmxAna6Vk+Bl7Tm3YKdvMr2Z8bp1H8HtyXHs8e+INdl3dsR+sJidsgv2i3VZjwmWsif2L4iDfvBnYXrQWLp/yFYiuP8PK+bZpQ==</latexit><latexit sha1_base64="izr5T8bnMAjJCNMWwvZUkQD63Cc=">AAACqHicbVJRSxtBEJ6cbbWptuqrL0dF6JPcCaW+FFqL4KPSRoXkCLN7c8ni3u6xOxcMR35CX1vwn/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTrTx4uWrza3X3TfbO2/f7e5tX3tbO0k9abV1twI9aWWox4o13VaOsBSabsTdt1a/mZDzypofPK0oK3FkVKEkcqC+T4anw93D5DiZR/wUpEtwCMu4HO51Hga5lXVJhqVG7/tpUnHWoGMlNc26g9pThfIOR9QP0GBJPmvmXmfxUWDyuLAuHMPxnH18o8HS+2kpQmaJPPbrWks+p/VrLk6zRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckyfC9SesqZUhilfaarhuWth71dpYmv1gj2j0CGd0UiZczNRzpp2Wv+KNQPRKo/qzI66g69FcHJu8mcvkMnX0lmKut27cOim/22Gbabru3sKrk+O04CvEtiCA3gPHyCFT/AFLuASeiBhBD/hF/yOPkb9SCz2HnWWH2AfViLK/wKlTdu9</latexit><latexit sha1_base64="izr5T8bnMAjJCNMWwvZUkQD63Cc=">AAACqHicbVJRSxtBEJ6cbbWptuqrL0dF6JPcCaW+FFqL4KPSRoXkCLN7c8ni3u6xOxcMR35CX1vwn/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTrTx4uWrza3X3TfbO2/f7e5tX3tbO0k9abV1twI9aWWox4o13VaOsBSabsTdt1a/mZDzypofPK0oK3FkVKEkcqC+T4anw93D5DiZR/wUpEtwCMu4HO51Hga5lXVJhqVG7/tpUnHWoGMlNc26g9pThfIOR9QP0GBJPmvmXmfxUWDyuLAuHMPxnH18o8HS+2kpQmaJPPbrWks+p/VrLk6zRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckyfC9SesqZUhilfaarhuWth71dpYmv1gj2j0CGd0UiZczNRzpp2Wv+KNQPRKo/qzI66g69FcHJu8mcvkMnX0lmKut27cOim/22Gbabru3sKrk+O04CvEtiCA3gPHyCFT/AFLuASeiBhBD/hF/yOPkb9SCz2HnWWH2AfViLK/wKlTdu9</latexit><latexit sha1_base64="xunXA1IUYOSicXgGkapNqZJQmpY=">AAACs3icbVFdSxtBFJ1sa2vTWrV99GUxCH2SXcHWF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNZc0KTHTMDSykAIoUL/Gg6PBVifZT2YRPwfpAnTYIs4H2637fm5FpdGQUOB9L01KympwJIXCabtfeSxB3MIQewEa0OizeqZ1Gu8FJo8L68IzFM/Ypx01aO8nmodKDTTyq7mGfCnXq6g4ymppyorQiPlHRaVisnGzeJxLh4LUJAAQTgatsRiBA0HhPEuTBIgRHhegPGa1loYwX1qqpplqbn8v00jWqjl7imFDPMWhNGdmLJ01zbX+Dav7vMk8mTPda/e/F0HJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6uD/TTgi6Rz8nXh6zrbYbvsC0vZN3bCfrJz1mWCDdkf9pfdRYdRL+JRPi+NWouez2wpIv0IK67czA==</latexit><latexit sha1_base64="xunXA1IUYOSicXgGkapNqZJQmpY=">AAACs3icbVFdSxtBFJ1sa2vTWrV99GUxCH2SXcHWF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNZc0KTHTMDSykAIoUL/Gg6PBVifZT2YRPwfpAnTYIs4H2637fm5FpdGQUOB9L01KympwJIXCabtfeSxB3MIQewEa0OizeqZ1Gu8FJo8L68IzFM/Ypx01aO8nmodKDTTyq7mGfCnXq6g4ymppyorQiPlHRaVisnGzeJxLh4LUJAAQTgatsRiBA0HhPEuTBIgRHhegPGa1loYwX1qqpplqbn8v00jWqjl7imFDPMWhNGdmLJ01zbX+Dav7vMk8mTPda/e/F0HJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6uD/TTgi6Rz8nXh6zrbYbvsC0vZN3bCfrJz1mWCDdkf9pfdRYdRL+JRPi+NWouez2wpIv0IK67czA==</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="HU6y5tdQYmCY7UB0YNHwaGEKd+A=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB0WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxVW35M=</latexit><latexit sha1_base64="xunXA1IUYOSicXgGkapNqZJQmpY=">AAACs3icbVFdSxtBFJ1sa2vTWrV99GUxCH2SXcHWF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pm5ubX+68rZyArvCKutuOHhU0mCXJCm8KR2C5gqv+e2PJn89RuelNZc0KTHTMDSykAIoUL/Gg6PBVifZT2YRPwfpAnTYIs4H2637fm5FpdGQUOB9L01KympwJIXCabtfeSxB3MIQewEa0OizeqZ1Gu8FJo8L68IzFM/Ypx01aO8nmodKDTTyq7mGfCnXq6g4ymppyorQiPlHRaVisnGzeJxLh4LUJAAQTgatsRiBA0HhPEuTBIgRHhegPGa1loYwX1qqpplqbn8v00jWqjl7imFDPMWhNGdmLJ01zbX+Dav7vMk8mTPda/e/F0HJmclfbECTr5ST4FXjO3fgJv9lBjfTVe+eg6uD/TTgi6Rz8nXh6zrbYbvsC0vZN3bCfrJz1mWCDdkf9pfdRYdRL+JRPi+NWouez2wpIv0IK67czA==</latexit>

v7<latexit sha1_base64="dB/GJtb3D49gFl2WcODK2B4yvg8=">AAACs3icbVFdSxtBFJ2sttVoW7WPfVkUwSfZLVT7UlCLtI9KGxWSJdyZvZsMzscyczcYlvyEvir6z/pvOpvYYqIHBg7nfsy59/JSSU9J8qcVLS2/ev1mZbW9tv723fuNza0LbysnsCOssu6Kg0clDXZIksKr0iForvCSX39r4pcjdF5a84vGJWYaBkYWUgAF6eeof9jf2En2kyni5yR9JDtHa9/vWcBZf7P10MutqDQaEgq876ZJSVkNjqRQOGn3Ko8liGsYYDdQAxp9Vk+9TuLdoORxYV14huKp+rSiBu39WPOQqYGGfjHWiC/FuhUVX7JamrIiNGL2UVGpmGzcDB7n0qEgNQ4EhJPBayyG4EBQWM9cJwFiiF8LUB6zWktDmM8NVdPUNbc38zKStWqmnmCYEE9wIM2pGUlnTbOtf83qHm8iT/pMdtu94yI4OTX5iwVo8oV0Erxq7s4duPF/m+Ga6eLtnpOLT/tp4OfhrAdshhX2kW2zPZayQ3bEfrAz1mGCDdhvdsvuos9RN+JRPkuNWo81H9gcIv0XqFLd9g==</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="PPwaTwye9fNx5nrXxNntNOXTfm0=">AAAConicbVFNT9tAEN24paQBCpx7sYgi9YRsLnBBAiqkHlOJkEiJC7PrcbJivWvtjlEjK3+Aa2/8M/5N10lAJGSklZ7efOybebxQ0lEUvTSCT5+3vmw3v7Z2dlt73/YPdm+dKa3AnjDK2AEHh0pq7JEkhYPCIuRcYZ8//Kzz/Ue0Thp9Q9MCkxzGWmZSAHmqe3fQjo6jeYQfQbwEbbaMu8PG8yg1osxRk1Dg3DCOCkoqsCSFwllrVDosQDzAGIceasjRJdVc5yzseCYNM2P90xTO2fcdFeTOTXPuK3OgiVvP1eSm3LCk7CyppC5KQi0WH2WlCsmE9dJhKi0KUlMPQFjptYZiAhYE+dOsTBIgJniegXKYVLnUhOnKUhXNVXPzd5VGMkYt2Cv0G+IVjqW+1o/SGl1f63VYNeJ15t2cWac1usy8kmudbmxAna6Vk+Bl7Tm3YKdvMr2Z8bp1H8HtyXHs8e+INdl3dsR+sJidsgv2i3VZjwmWsif2L4iDfvBnYXrQWLp/yFYiuP8PK+bZpQ==</latexit><latexit sha1_base64="54esQxMz4DTDx/N/CM7J5qmXIdY=">AAACqHicbVJRSxtBEJ5ctbVRW+1rXw5F8EnuBLEvhVYR+mhpo0JyhNm9uWRxb/fYnQuGIz+hry30n/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTvRiY/Plq63X3e2d3Tdv9/Z3brytnaSetNq6O4GetDLUY8Wa7ipHWApNt+L+stVvJ+S8suY7TyvKShwZVSiJHKhvk+H5cO8wOUnmET8F6RIcwjKuh/ud34Pcyrokw1Kj9/00qThr0LGSmmbdQe2pQnmPI+oHaLAknzVzr7P4KDB5XFgXjuF4zj6+0WDp/bQUIbNEHvt1rSWf0/o1Fx+yRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckwfC9SesqZUhilfaarhuWthH1ZpYmv1gr2g0CFd0EiZKzNRzpp2Wv+KNQPRKo/qzI66g89FcHJl8mcvkMnX0lmKut27cOim/22Gbabru3sKbk5P0oC/JrAF7+EAjiGFc/gEX+AaeiBhBD/gJ/yKzqJ+JBZ7jzrLD/AOViLK/wKjC9u8</latexit><latexit sha1_base64="54esQxMz4DTDx/N/CM7J5qmXIdY=">AAACqHicbVJRSxtBEJ5ctbVRW+1rXw5F8EnuBLEvhVYR+mhpo0JyhNm9uWRxb/fYnQuGIz+hry30n/XfdC9Ji4kOLHx83+zsNzMrKq08J8mfTvRiY/Plq63X3e2d3Tdv9/Z3brytnaSetNq6O4GetDLUY8Wa7ipHWApNt+L+stVvJ+S8suY7TyvKShwZVSiJHKhvk+H5cO8wOUnmET8F6RIcwjKuh/ud34Pcyrokw1Kj9/00qThr0LGSmmbdQe2pQnmPI+oHaLAknzVzr7P4KDB5XFgXjuF4zj6+0WDp/bQUIbNEHvt1rSWf0/o1Fx+yRpmqZjJy8VBR65ht3DYe58qRZD0NAKVTwWssx+hQchjPSiWJckwfC9SesqZUhilfaarhuWthH1ZpYmv1gr2g0CFd0EiZKzNRzpp2Wv+KNQPRKo/qzI66g89FcHJl8mcvkMnX0lmKut27cOim/22Gbabru3sKbk5P0oC/JrAF7+EAjiGFc/gEX+AaeiBhBD/gJ/yKzqJ+JBZ7jzrLD/AOViLK/wKjC9u8</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="Cume9FAzFMWPUzaghgZgFj9jRgk=">AAACs3icbVFbSxtBFJ5sbxrbqu2jL0tF6JPsCl5eCmqR9lGx0UCyhDOzZ5PBuSwzZ4NhyU/oa0v7f/oj+m86m2gx0QMDH9+5zHfOx0slPSXJ31b07PmLl69WVttrr9+8Xd/YfHflbeUEdoRV1nU5eFTSYIckKeyWDkFzhdf85nOTvx6j89KabzQpMdMwNLKQAihQl+PB4WBjO9lNZhE/Bukd2D5e+/Jr6083Px9stn73cysqjYaEAu97aVJSVoMjKRRO2/3KYwniBobYC9CARp/VM63TeCcweVxYF56heMY+7KhBez/RPFRqoJFfzjXkU7leRcVRVktTVoRGzD8qKhWTjZvF41w6FKQmAYBwMmiNxQgcCArnWZgkQIzwUwHKY1ZraQjzhaVqmqnm9naRRrJWzdlTDBviKQ6lOTNj6axprnU/rO7zJvNgznSn3T8pgpIzkz/ZgCZfKifBq8Z37sBN/ssMbqbL3j0GV3u7acAXwdYDNo8VtsU+sI8sZYfsmH1l56zDBBuy7+wH+xntR72IR/m8NGrd9bxnCxHpfxMJ35I=</latexit><latexit sha1_base64="xs2o3//PTicbHGG4n+u1Ynf46qI=">AAACs3icbVFdSxtBFJ1sa2vT2mr76MtiEPoku4LVF8FahD4qNSokS7gzezcZnI9l5m5oWPIT+lrBf+a/cTZJi4leGDic+zHn3sNLJT0lyUMrevV67c3b9Xft9x82Pn7a3Pp85W3lBHaFVdbdcPCopMEuSVJ4UzoEzRVe89sfTf56jM5Lay5pUmKmYWhkIQVQoH6NB4eDzU6yl8wifg7SBeiwRZwPtlr3/dyKSqMhocD7XpqUlNXgSAqF03a/8liCuIUh9gI0oNFn9UzrNN4NTB4X1oVnKJ6xTztq0N5PNA+VGmjkV3MN+VKuV1FxlNXSlBWhEfOPikrFZONm8TiXDgWpSQAgnAxaYzECB4LCeZYmCRAjPC5AecxqLQ1hvrRUTTPV3P5eppGsVXP2FMOGeIpDac7MWDprmmv9G1b3eZN5Mme62+5/L4KSM5O/2IAmXyknwavGd+7ATf7LDG6mq949B1f7e2nAF0nn5NvC13W2zXbYV5ayQ3bCfrJz1mWCDdkf9pfdRQdRL+JRPi+NWoueL2wpIv0IKWHcyw==</latexit>

Fig. 1: Graph with triangle {v1, v7, v8} highlighted andcorresponding representation as a sparse matrix.

achieving high levels of performance in parallel also requiresan approach to load balancing that interacts harmoniously withall of the other aspects of the algorithm.

II. BACKGROUND

A. Triangle Counting

Given an undirected graph G = {V,E}, a triangle is atriple of vertices vi, vj , vk, each in the vertex set V such thatthe three edges {vi, vj}, {vj , vk} and {vk, vi} are in the edgeset E. If we further require that i < j < k, each such tripleis unique. Algorithms for triangle counting seek to find thecardinality of the set of unique triangles.

An exhaustive (and inefficient) algorithm is simply to ex-amine all triples of edges:α← 0for each edge {vi, vj} ∈ E, i < j do

if ∃k < i s.t. {vk, vi} ∈ E and {vk, vj} ∈ E thenα← α+ 1

There are some important aspects of the problem structurethat can be exploited to improve computational complexity oftriangle counting; a thorough exposition of triangle countingalgorithms can be found in [2]. Despite AL’s (exaggerated)reputation for antipathy towards graph algorithms based onlinear algebra, sparse matrix interpretations of triangle count-ing provide valuable intuition about these exploits.

In sparse matrix terms, a triangle exists if there are entries(i, j), (j, k), and (k, i) in the sparse matrix representationof the graph. Since we have i < j < k, we only need toconsider one triangle of the sparse matrix (we consider theupper triangle as shown in Figure 1, in which case we lookfor entries (i, j), (j, k), and (i, k)). That is, given a vertex i,we consider all of the edges (i, j) to its neighbors with i < j.For each such neighbor j, we look at all of its neighbors k aswell as the remaining neighbors of i. If the same neighbor kappears for both i and and j, then there is a triangle {i, j, k}.

978-1-7281-9219-2/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: University of Washington Libraries. Downloaded on June 30,2021 at 21:08:37 UTC from IEEE Xplore. Restrictions apply.

We thus have the following (the count is stored in α):α← 0for i = 0, 1 . . . |V | − 1 do

for Each neighbor j of i, j > i doα← α+ |Nb(i, j)⋃Nb(j)|

Here, |Nb(i, j)⋃Nb(j)| means the number of intersectionsbetween the neighbors of i (that are greater than j) and theneighbors of j.

B. Implementation

This triangle counting algorithm can be realized in C++ asshown in Figure 2. The details are explained in Section IV butof note here is that the implementation is essentially a directtransliteration of the algorithm. Moreover, the implementation(as does the algorithm) makes no assumptions about thespecific storage format of the graph – only that neighbor listsare stored as “forward ranges”, that all of the neighbor listsare stored as a “random-access range”, and that within eachneighbor list, the vertex ids are stored in sorted order.

C. Shared-memory Triangle Counting Algorithms

A plethora of works have been done in the context oftriangle counting algorithms. For example, Shun et al. [3]presented cache-oblivious shared-memory triangle countingalgorithms based on Latapy’s sequential algorithms [2]. Sub-sequent work by Parimalarangan et al. [4] classified trianglecounting algorithms into adjacency intersection (AI)-basedand adjacency marking (AM)-based methods. They applied acanonical representation of the vertex triple (u, v, w), based ondegree ordering, to avoid counting triangles more than once.They enumerate possible triangle counting algorithms basedon degree ordering. Similar approaches have been taken in [5]too. However, partitioning of work across different threads wasleft to the OpenMP scheduler (dynamic scheduling) in previ-ous works. In this paper, we show that not only preprocessingsuch as vertex ordering is helpful for optimal work scheduling,but also distributing the workload in a cyclic manner resultsin the best performance of a triangle counting algorithm foran interesting set of input graphs.

In addition to the vertex-centric approaches to trianglecounting, efficient linear algebra-based triangle counting ((L×L). ∗ L) have also been proposed ( [6]–[9]). However, linearalgebra-based triangle counting imposes an additional memoryrequirement due to the temporary storage to compute inter-mediate sparse matrix-sparse matrix multiplication. Maskingand specialized fusion operation are required too. Furthermore,in [8], distribution of work in terms of greedy block parti-tioning, similar to our balanced block partitioning, has beenconsidered only with Cilk.

III. HIGH PERFORMANCE TRIANGLE COUNTING

While many triangle counting implementations rely on thebasic idea shown in Figure 2, high performance trianglecounting on large graphs will need additional improvementsfor reasonable performance. We aim to incorporate these

improvements while not straying too far from the simplicityof Figure 2.

One such improvement is to triangularize the adjacencymatrix. The information required to count all triangles isavailable in either the upper or lower triangular sections ofthe adjacency matrix so using the full adjacency matrix willrequire more work than necessary. Instead one can preprocessa graph to contain only successor edges, edges from vertices toneighbor vertices with a higher index. This corresponds to theupper triangular portion of the adjacency matrix. Alternativelyone can use predecessor edges which correspond to the lowertriangular portion of the adjacency matrix.

In order to minimize the total amount of work, it is oftenimportant to relabel vertices according to vertex degree. Thisis equivalent to permuting the rows and columns of thematrix (aka “diagonal pivoting”). While this can reduce thetotal amount of sequential work, the net effect is to groupall of the high-degree vertices together, which can interferewith partition-based load balancing (the highest-degree nodesend up in the same partition). However, different workloaddistributions can be used to improve load balancing.

In the rest of this section we explore these ideas andillustrate their benefits on some smaller test graphs from theSuiteSparse matrix collection [10], whose sizes are shown inFigure 4. We analyze these approaches using the number ofelement comparisons in the set intersection step of trianglecounting as a proxy for total work. Full timing experimentswith large-scale graphs are given in Section Section V.

A. Degree Reordering

The goal of ordering graphs for efficient triangle countingis closely related to determining orderings for minimizingfill in sparse direct methods for LU factorization [11]. Pop-ular heuristic algorithms for this include Reverse Cuthill-McKee [12], Modified Minimum Degree [13], and Approx-imate Minimum Degree [14]. Beyond fill minimization, or-dering techniques have been applied to improve the perfor-mance of some graph kernels (such as BFS, Single-sourceshortest Paths (SSSP), PageRank and subgraph counting) in adistributed setting [15]. The effects of degree-based orderingon the performance of triangle counting have been noted bynumerous authors (e.g., [2]–[4], [16]).

Reordering vertices can affect how many vertex compar-isons are performed when intersecting two neighbor lists. Thenumber of comparisons is a good proxy for the total amountof work required, thus reducing it can have a great affect on(sequential) total solution time. When processing successorlists (upper triangle shape), we relabel vertices so that they arein ascending order by degree (the highest-degree vertices goat the end). When processing predecessor lists (lower triangleshape), we relabel vertices so that the are in descending orderby degree (the highest-degree vertices go at the beginning).

To illustrate the effect of reordering, Figure 4 shows theamount of work saved by degree reordering as a fraction ofthe work without reordering for an assortment of several testgraphs. Note that these are results from triangle counting on

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


template <typename RandomAccessRange>size_t triangle_count(RandomAccessRange G){

size_t alpha = 0; // α← 0for (auto i = G.begin(); i != G.end(); ++i) // for i = 0, 1, . . . V − 1for (auto j = (*i).begin(); j != (*i).end(); ++j) // for each neighbor j of ialpha += intersection_size(*i, G[*j]); // α← α+Nb(i, j)

⋃Nb(j)

return alpha;}

Fig. 2: Triangle counting algorithm in C++.

(a) No Reordering

(b) Reorder Ascending Degree

Fig. 3: Web-Google Spyplots and Degree Distributions: Spyplots of the web-Google graph are shown (a) before and(b) after reordering by ascending degree along with their corresponding predecessor and successor degree distributions.Predecessor/Successor degrees count number of entries per row of the upper/lower triangular matrix respectively. The degreedistributions of (b) are only shown for the region on interest indicated in the spyplot.

Graph |V | |E| Work With ReorderWork Without Reorder

as-Skitter 1.70M 11.1M 81.8%com Amazon 335K 926K 8.09%com Youtube 1.13M 2.99M 83.5%kron g500-logn19 524K 21.8M 0.00% (2.75e-11)soc-LiveJournal1 4.85M 34.5M 100%web-Google 916K 2.55M 75.4%wiki-Talk 2.39M 2.51M 89.7%

Fig. 4: Successor Degree Reorder Improvement of UpperTriangle: The ratio of total work with degree reorderingcompared to no reordering for several SuiteSparse graphs.

successor lists, and use ascending degree reordering. Severalgraphs require 80%-90% of the original work, though some-times the benefit is much greater in the case of Kroneckergraphs, and sometimes there is little benefit as with soc-LiveJournal1. In general, the more skewed the degree distri-

bution of a graph is (the more like a power-law graph it is),the more benefit reordering will provide.

To more fully illustrate the effect of reordering on graphstructure, we show spyplots of the same graph before andafter ascending degree reordering (Figure 3(a) and Figure 3(b),respectively. Spyplots on the left indicate the sparsity patterns,with the reordered graph having most of its entries shifted tothe lower right. Predecessor and successor degree distributionsare shown before and after reordering. These degree plotsonly show the window of interest and are scaled differentlyfor visual clarity. Note that the average degree is the sameregardless of which degree list we use or whether we reorderor not. This makes sense as the information needed for trianglecounting is preserved whether or not we reorder and whetherwe use successor or predecessor information. However theseveral very high degree vertices in both predecessor andsuccessor degree distributions of the unlabeled graph can hurt

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


performance and is why reordering is often useful. Reorderingin the wrong direction can be even more harmful. If we tryto use the predecessor list to count triangles after relabelingby ascending degree as seen in Figure 3(b), the maximumdegree is every higher. There is a clear benefit from usingthe successor list after reordering by ascending degree, as itsmax degree is much lower compared to both the unlabeleddegree distributions and the predecessor list of the reorderedgraph. If we had instead reordered by descending degree, thepredecessor distribution would have a smaller max degree anda smoother distribution. The rest of the paper assumes the useof successor lists unless otherwise stated.

While degree ordering has the effect of reducing the totalamount of sequential work, it also increases the difficulty ofload balancing in parallel. This is due to the very skewedtail seen in the successor distribution of Figure 3(b) andthe very dense lower right region of the associated spyplot.If some work threads are assigned vertices at the tail ofthis distribution, they will take much longer to process theirvertices’ neighbor lists than other threads. We discuss potentialways to avoid this below.

B. Parallelization

In this paper we consider very simple parallel extensionsof Figure 2, and we are primarily interested in how to dividethe vertex neighbor lists (matrix rows) across multiple threads.It is well known that there are multiple ways to assign rows tothreads (or rows to processors in a distributed setting) for betterload balancing, yet we have found few results of different rowdistribution strategies presented in isolation. The most naturalneighbor list distribution is a simple block partition, assigningthe first n/threads rows to the first thread and so on. Thisoften leads to poor load balancing, with the smaller but muchdenser rows near the tip of the triangular adjacency matrixcontaining a disproportionately large amount of work. Thepoor load balancing of the simple block distribution across64 threads for the web-Google graph can be seen in Figure 5.The maximum work assigned to a thread is several orders ofmagnitude larger than the minimum work done by a thread.

To avoid this, the rows could be partitioned non-uniformlydepending on their number of entries. In this balanced blockdistribution, the rows owned by a thread would still becontiguous but the number of rows per thread would vary.The effect of this improvement on the web-Google graphcan be seen in Figure 5. This distribution is an improvementover the simple block distribution, but an order of magnitudeload balance remains between the minimum and maximumwork threads, due to a remaining uneven distribution of highsuccessor degree vertices. To further improve load balance weutilize a cyclic row distribution, where each thread t operateson rows t, threads + t, (2 × threads) + t, and so on. Thiscyclic strategy has similarities to a 2D distributed memoryimplementation [17], where it can be inferred but is not shownexplicitly that cyclic load balancing improves performance.For the web-Google graph, this leads to near optimal loadbalancing as indicated by the flat line in Figure 5.

0 20 40 60Thread

104

105

106

Wor

k Lo

ad

BlockBalanced BlockCyclic

Fig. 5: Web-Google Workload Distributions: The amount ofwork required for each of 64 threads under three differentworkload distributions. The flat line for cyclic distributionindicates near optimal workload balancing.

as-S

kitter

com_A

mazon

com_Y

outub

ekro

n_g5

00-lo

gn19

soc-L

iveJo

urna

l1web

-Goo

glewiki

-Talk

0.0

0.2

0.4

0.6

Wor

k R

elat

ive

to B

lock

Balanced BlockCyclic

Fig. 6: Maximum Workload Over Threads: The benefit ofbalanced block and cyclic neighbor list distributions relativeto a simple block workload distribution for several SuiteS-parse graphs. Shorter bars indicate less work required by thebottleneck thread.

As the thread with the most assigned work becomes theperformance bottleneck, we track the max thread work over 64threads while triangle counting on several smaller graphs andpresent the results in Figure 6. This bar plot uses the simpleblock distribution as a baseline and indicates the improvementof balanced block and cyclic methods. Note that using a cyclicdistribution is often a small fraction of the max work overthreads compared to block distributions.

IV. IMPLEMENTATIONS

In this section we present descriptions of different trianglecounting implementations based on three different C++ paral-lelization strategies, std::async, tbb::parallel_for,and std::for_each with parallel execution policies. Weapply these strategies to the outer level of parallelism, splittingup the work of processing different neighbor lists. In all

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


the implementations, we try to rely on generic, reusableprogramming as much as possible, including STL algorithmsand data structures. For instance we choose to use std::vectorinstead of writing our own, even if there is some performanceto be gained. There is a potential extra level of parallelism tobe gained by using parallel execution policies inside the set in-tersections of all the implementations discussed below. Whilewe hope to develop a fast triangle counting implementation,the main goal is to understand the performance differences ofdifferent algorithmic techniques and parallelization strategies.See Appendix Figures 12-14 for example NWGraph code.

A. Implementations based on std::async

The first parallelization strategy we consider is to distributework among threads by passing lambda functions to async.The lambda function uses the size of the graph and thenumber of threads to determine the neighbor lists owned by thecurrent thread, based on the different neighborlist distributionstrategies discussed in Section III. After threads return theirtriangle count, the sum is reduced with std::future.

B. Implementations based on tbb::parallel_reduce

Our next triangle counting implementation uses ThreadingBuilding Blocks (TBB)’s parallel_reduce to split theneighbor lists among threads and to accumulate the numberof triangles. This parallel reduce takes as an argument a rangeof vertices for each thread to operate over. One could passin a simplistic block range, but again we use a cyclic rangefor better load balancing. Since the TBB implementation hasless explicit thread control, we develop a novel cyclic rangeadapter that TBB uses to cyclically assign work to threads.

C. Implementations based on std::for_each and C++Parallel Execution Policies

The last implementation we consider uses the C++ built-in std::for_each function to process the neighbor listsof every vertex. Instead of explicitly dividing these neighborlists among threads, for_each takes an optional executionpolicy for processing the lists in parallel. Note that using thisstrategy is a more hands off approach as a programmer has nocontrol how for_each ultimately splits up the workload.

V. EXPERIMENTAL SETUP AND RESULTS

To see the benefits of the techniques discussed in theprevious section, we count the number of triangles in thegraphs described in the GAP benchmark suite [16] and collecttiming results. The GAP benchmark is a set of five test graphsand six graph kernels, including triangle counting, designed totest the performance of different graph processing frameworks.The smaller graphs used for the work counting experimentsin Section III were chosen to have similar properties to theGAP graphs. GAP also includes a reference implementation,included in our performance results for comparison, whichparallelizes the processing of neighbor lists with OpenMPand dynamic scheduling. We choose to omit the GAP-roadnetwork as it is an outlier for this study in terms of size and

GAP-kron

GAP-twitte

r

GAP-urand

GAP-web

100

101

102

Spe

edup

of R

eord

erin

g

Fig. 7: Reordering on GAP Graphs. Triangle counting speedupby reordering is shown relative to unordered on a log scale.All graphs shown benefit except for GAP-urand.

Gap |V | |E| Triangle NWGraph GAPGraphs Count Relabel(s) Relabel(s)web 50.6M 965M 84.91B 1.06 11.1twitter 61.6M 734M 34.83B 1.53 15.4kron 134M 2.11B 106.9B 2.97 24.8urand 134M 2.15B 5378 3.79 26.5

Fig. 8: Triangle and Relabel Information for GAP Graphs

5.060e7 5.0605e7 5.061e7Vertex

0

1000

2000

3000

4000

Suc

cess

or D

egre

e

Fig. 9: GAP-Web Partial Successor Degree Distribution. Acharacteristic double spike of large degree neighbor lists is achallenge to load balance.

the number of triangles. The included graphs and their sizesare shown in Figure 8. Our testing architecture is an Intel XeonGold 6130 CPU with 64 logical cores. For these experimentswe utilize all 64 cores. We compiled all implementations,including the GAP reference, with GNU GCC 9.2.0.

Not all graphs will benefit equally from graph reorderingand the overhead of reordering might be costly compared tothe actual triangle counting. In practice one might employcheap heuristics to decide whether reordering overhead isworthwhile. To demonstrate the effectiveness of reorderingon the GAP graphs, we measure triangle counting time be-fore and after reordering and present the reordering speedupin Figure 7. These results are based on the cyclic asyncimplementation. Kron sees nearly an order of magnitudeimprovement in solve time while the skewed degree web

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


GAP-kron

GAP-twitte

r

GAP-urand

GAP-web

0.0

0.5

1.0

Spe

edup

Rel

ativ

e to

GA

PBlockBalanced BlockCyclic

(a) Row Distribution (All Async)

GAP-kron

GAP-twitte

r

GAP-urand

GAP-web

0.0

0.5

1.0

asyncTBBfor_each

async Parallel IntersectionTBB Parallel Intersectionfor_each Parallel Intersection

(b) Parallelization Strategy

Fig. 10: GAP Comparison: The speedup (or slowdown) of several implementations is shown relative to the GAP referenceimplementation on the GAP graphs. Implementations above the dotted line run faster than the GAP reference implementation. (a)compares row distribution strategies of different async implementations. (b) compares different C++ parallelization strategies.

and twitter graphs see two orders of magnitude improvement.Urand sees a slight slowdown due to reordering since therandom nature of the graph already had a smooth degreedistribution to begin with.

However, to provide a level playing field for the rest of thealgorithmic modifications we relabel all graphs by ascendingdegree and report relabel time along with the number oftriangles in Figure 8. The relabeling time of the GAP referenceimplementations typically an order of magnitude slower thanour implementation because we relabel an edge list beforecompressing to compressed sparse row (CSR) format.

In order to demonstrate the effect each of the algorithmictechniques and C++ parallelization strategies, we tested multi-ple implementations and summarize results in Figure 10. Eachbar in this figure indicates the speedup over the GAP referenceimplementation (GAP time / implementation time), with barsabove the dotted line indicating improvement over GAP.Figure 10(a) compares the first three implementations whichare based on std::async and demonstrate the differentneighbor list distribution strategies in Section III. Figure 10(b)compares the three outer level parallelization strategies de-scribed in Section IV. This figure also includes results fromadding an inner level of parallelism with a parallel executionpolicy inside the set intersection. Note that Figure 10(a) onlyincludes sequential STL intersections and Figure 10(c) onlyincludes cyclic distribution for async and TBB. We includethe timing numbers behind these plots in Appendix Figure 11.

VI. EXPERIMENTAL ANALYSIS AND DISCUSSION

Figure 10(a) indicates that cyclic distributions often lead toimproved triangle solve time compared to block distributionsdue to better load balancing. GAP-urand is an outlier as it hasa more balanced degree distribution to begin with so simple

block partitioning has decent load balance. While it mightbe possible, knowing the cyclic stride in advance, to createan artificial graph which causes poor load balancing for acyclic distribution, such graphs are unlikely to occur naturally.A cyclic distribution will almost never lead to poor loadbalancing and will often significantly improve performance,assuming the number of compute resources is not significantlylarger than the number of high degree vertices.

Figure 10(b) suggests that cyclic implementations basedon std::async and tbb::parallel reduce are quite competitive,with async slightly outperforming on GAP-web. The imple-mentation based on for_each performs worse, significantlyworse in the case of GAP-web. We avoid speculating onhow for_each assigns neighbor lists to threads “behind thecurtain”. Performance is typically better than for the blockand balanced block async implementations shown in Fig-ure 10(a), so it is doing something reasonable. As the parallelfor_each implementation is perhaps the most ideal in termsof relying on STL features, we would like to better understandwhat is required to make its performance more comparable tothe other parallelization strategies.

The cyclic distribution strategy closely matches the resultsof the GAP reference implementation on three of the graphsbut not GAP-web where the cyclic distribution seems tooutperform GAP’s block distribution. This led us to furtherinvestigate the successor degree distribution of GAP-web afterreordering and we found two interesting spikes near the tailend of the distribution shown in Figure 9

None of the outer parallelization strategies benefit fromparallel set intersections as evidenced by the dashed barsin Figure 10(b). Perhaps with good load balancing of all 64threads, we should not expect much benefit of using a parallelexecution policy inside the set intersection.

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


REFERENCES[1] A. Azad et al., Evaluation of graph analytic frameworks using the GAP

benchmark suite, (to appear in) Proc. IEEE Int. Symp. on WorkloadCharacterization, 2020.

[2] M. Latapy Main-memory triangle computations for very large (sparse(power-law)) graphs, Theoretical Computer Science 407.1-3 (2008),pp. 458–473.

[3] J. Shun and K. Tangwongsan, Multicore triangle computations withouttuning, in Proc. IEEE Int. Conf. on Data Engineering, Seoul, KOR,2015, pp. 149–160.

[4] S. Parimalarangan, G. M. Slota, and K. Madduri, Fast parallel graphtriad census and triangle counting on shared-memory platforms, inProc. IEEE Int. Parallel and Distributed Processing Symp. Workshops,Lake Buena Vista, FL, USA, 2017, pp. 1500–1509.

[5] A. S. Tom, N. Sundaram, N. K. Ahmed, S. Smith, S. Eyerman, M.Kodiyath, I. Hur, F. Petrini, and G. Karypis, Exploring optimizationson shared-memory platforms for parallel triangle counting algorithms,in Proc. IEEE High Performance Extreme Computing Conf., Waltham,MA, USA, 2017, pp. 1–7.

[6] J. Kepner and J. Gilbert, Graph algorithms in the language of linearalgebra, SIAM, 2011.

[7] M. M. Wolf, M. Deveci, J. W. Berry, S. D. Hammond, and S.Rajamanickam, Fast linear algebra-based triangle counting with Kokkoskernels, in Proc. IEEE High Performance Extreme Computing Conf.,Waltham, MA, USA, 2017, pp. 1–7.

[8] A. Yasar, S. Rajamanickam, M. Wolf, J. Berry, and U. V. Catalyurek,Fast triangle counting using Cilk, in Proc. IEEE High PerformanceExtreme Computing Conf., Waltham, MA, USA, 2018, pp. 1–7.

[9] A. Azad, A. Buluc, and J. Gilbert, Parallel triangle counting andenumeration using matrix algebra, in Proc. IEEE Int. Parallel and Dis-tributed Processing Symp. Workshop, Hyderabad, IND, 2015, pp. 804–811.

[10] T. Davis and Y. Hu, The University of Florida sparse matrix collection,ACM Transactions on Mathematical Software 38.1 (2011), pp. 1–25.

[11] A. George and J. W. Liu, Computer solution of large sparse positivedefinite, Pretince Hall, 1981.

[12] E. Cuthill and J. McKee, Reducing the bandwidth of sparse symmetricmatrices, in Proc. of ACM National Conf., New York, NY, USA, 1969,pp. 157–172.

[13] J. W. H. Liu, Modification of the minimum-degree algorithm by multipleelimination, ACM Transactions on Mathematical Software 11.2 (1985),pp. 141–153.

[14] P. R. Amestoy, T. A. Davis, and I. S. Duff, An approximate minimum de-gree ordering algorithm, SIAM J. on Matrix Analysis and Applications17.4 (1996), pp. 886–905.

[15] G. M. Slota, S. Rajamanickam, and K. Madduri, Order or shuffle:Empirically evaluating vertex order impact on parallel graph compu-tations, in Proc. IEEE Int. Parallel and Distributed Processing Symp.Workshops, Lake Buena Vista, FL, USA, 2017, pp. 588–597.

[16] S. Beamer, K. Asanovic, and D. Patterson, The GAP benchmark suite,2015, arXiv:1508.03619.

[17] A. S. Tom and G. Karypis, A 2D parallel triangle counting algorithmfor distributed-memory architectures, in Proc. ACM Int. Conf. ParallelProcessing, Kyoto, JPN, 2019, pp. 1–10.

VII. APPENDIX

GAP GAP Reference NWGraphGraphs Block Balanced Block Cyclic

Sequential Intersection Sequential Intersection Sequential Intersection Parallel Intersectionasync async async TBB for_each async TBB for_each

web 7.71 54.3 27.7 5.72 6.13 15.7 6.11 6.39 14.0twitter 45.4 1400 151 43.5 43.2 54.8 44.3 43.5 53.0kron 277 7750 509 290 285 301 291 287 314urand 15.0 19.6 16.1 15.1 14.6 20.8 14.2 14.4 20.2

Fig. 11: Triangle Counting Timing in (s) for All Implementations Shown in Figure 10.

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


template <class Op>size_t async_helper(size_t threads, Op&& op){

vector<future<size_t>> futures(threads);for (size_t tid = 0; tid < threads; ++tid)

futures[tid] = async(launch::async, op, tid);

size_t t = 0;for (auto&& f : futures)t += f.get();

return t;}


return async_helper(threads, [&](size_t tid) {size_t alpha = 0;for (auto i = G.begin(); i != G.end(); ++i)

for (auto j = (*i).begin(); j != (*i).end(); ++j)alpha += intersection_size(*i, G[*j]);

return alpha;});

}

Fig. 12: Parallel Triangle Counting with std::async.

template <typename RandomAccessRange>size_t triangle_count(RandomAccessRange G, size_t stride){

return nwgraph::parallel_for(nwgraph::cyclic(G, stride), [&](auto&& i) {size_t alpha = 0;for (auto j = (*i).begin(); j != (*i).end(); ++j)alpha += intersection_size(*i, G[*j]);

return alpha;}, plus{}, 0.0);

}

Fig. 13: Parallel Triangle Counting with TBB.


atomic<size_t> t = 0;for_each(execution::par, G.begin(), G.end(), [&](auto&& i){

size_t alpha = 0;for (auto j = (*i).begin(); j != (*i).end(); ++j)alpha += intersection_size(*i, G[*j]);

t += alpha;});return t;

}

Fig. 14: Parallel Triangle Counting with std::for_each.

978-1-7281-9219-2/20/$31.00 ©2020 IEEE


Triangle Counting with Cyclic Distributions - NSF PAR

Documents

Transcript of Triangle Counting with Cyclic Distributions - NSF PAR