Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bench): PoC of integration with zkalc #425

Merged
merged 12 commits into from
Jul 12, 2024
Merged

feat(bench): PoC of integration with zkalc #425

merged 12 commits into from
Jul 12, 2024

Conversation

mratsim
Copy link
Owner

@mratsim mratsim commented Jul 11, 2024

This PR provides integration with zkalc from @mmaker:

The strategy adopted is to compile a binary that then can be called with curve parameter and vartime or not. It will then output the benchmarks in console and directly in zkalc json format to avoid implementing a parser in zkalc repo.

Current bench Output

with the initial commit POC

image

Statistics

We target a sampling time of 5s per benchmark.
Subroutines that take less than 10ms are collated into batches of about ~10ms
to amortize getClock/syscall/vDSO noise, see https://gms.tf/on-the-costs-of-syscalls.html

TODO

  • Refactor to allow G2 benches on relevant curves
  • Json output
  • Add to CI
  • nimble task
  • Add pairing and multi-pairing
  • Add GT mul and exponentiation (i.e. cyclotomic_exp)

@mratsim
Copy link
Owner Author

mratsim commented Jul 12, 2024

PR is finalized. I won't add the mul_GT exp_GT and multiexp_GT yet as:

  • exp_GT is not implemented, there is cyclotomic_exp because Gt implies cyclotomic (but cyclotomic does not imply Gt)
  • multiexp_GT is not implemented

Usage

Requires nim v2.0.8 and Clang.

Building

git clone https://github.com/mratsim/constantine
cd constantine
nimble make_zkalc

Running examples, if curve is unspecified it runs BN254 and file output is
working_directory/constantine-bench-zkalc-${curve}-${YYYY-MM-DD}--${HH-mm-ss}.bench.json

bin/constantine-bench-zkalc --curve=BN254_Snarks
bin/constantine-bench-zkalc --curve=BLS12_381 --o=myoutputfile.json

Curves currently supported are

  • BN254_Snarks
  • Pallas
  • Vesta
  • BLS12-377
  • BLS12-381

Output

Fr and G1

image

G2 and pairings

image

Whole text

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔽r Addition                                        BN254_Snarks  1000000000.000 ops/s             1.0 ns/op (avg)    ± 0.0% (coef var)     500 iterations of 510138 operations
𝔽r Multiplication                                  BN254_Snarks    71428571.429 ops/s            14.0 ns/op (avg)    ± 0.0% (coef var)     500 iterations of 389497 operations
𝔽r Inversion                    | vartime          BN254_Snarks     1313432.209 ops/s           761.4 ns/op (avg)    ± 0.3% (coef var)     500 iterations of  11894 operations
𝔽r Sum of products of size 2                       BN254_Snarks    55623539.882 ops/s            18.0 ns/op (avg)    ± 0.8% (coef var)     500 iterations of 293585 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₁ Addition                     | vartime          BN254_Snarks    10578428.469 ops/s            94.5 ns/op (avg)    ± 0.6% (coef var)     500 iterations of  86116 operations
𝔾₁ Scalar Multiplication        | vartime          BN254_Snarks       37243.379 ops/s         26850.4 ns/op (avg)    ± 0.3% (coef var)     500 iterations of    305 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating 2097152 (coefs, points) pairs ... in 2314.675 ms
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₁ MSM         2,  16 threads   | vartime          BN254_Snarks       22266.465 ops/s         44910.6 ns/op (avg)    ± 2.0% (coef var)     500 iterations of    203 operations
𝔾₁ MSM         4,  16 threads   | vartime          BN254_Snarks       22943.272 ops/s         43585.8 ns/op (avg)    ± 0.6% (coef var)     500 iterations of    222 operations
𝔾₁ MSM         8,  16 threads   | vartime          BN254_Snarks       19427.826 ops/s         51472.6 ns/op (avg)    ± 1.1% (coef var)     500 iterations of    192 operations
𝔾₁ MSM        16,  16 threads   | vartime          BN254_Snarks       16025.582 ops/s         62400.2 ns/op (avg)    ± 0.7% (coef var)     501 iterations of    142 operations
𝔾₁ MSM        32,  16 threads   | vartime          BN254_Snarks       11596.374 ops/s         86233.9 ns/op (avg)    ± 0.6% (coef var)     502 iterations of    113 operations
𝔾₁ MSM        64,  16 threads   | vartime          BN254_Snarks        7776.372 ops/s        128594.7 ns/op (avg)    ± 0.5% (coef var)     503 iterations of     76 operations
𝔾₁ MSM       128,  16 threads   | vartime          BN254_Snarks        4728.299 ops/s        211492.5 ns/op (avg)    ± 1.1% (coef var)     504 iterations of     46 operations
𝔾₁ MSM       256,  16 threads   | vartime          BN254_Snarks        2710.250 ops/s        368969.7 ns/op (avg)    ± 1.6% (coef var)     507 iterations of     27 operations
𝔾₁ MSM       512,  16 threads   | vartime          BN254_Snarks        1724.345 ops/s        579930.2 ns/op (avg)    ± 0.8% (coef var)     502 iterations of     17 operations
𝔾₁ MSM      1024,  16 threads   | vartime          BN254_Snarks        1062.992 ops/s        940740.8 ns/op (avg)    ± 1.3% (coef var)     542 iterations of      9 operations
𝔾₁ MSM      2048,  16 threads   | vartime          BN254_Snarks         655.463 ops/s       1525638.8 ns/op (avg)    ± 1.8% (coef var)     539 iterations of      6 operations
𝔾₁ MSM      4096,  16 threads   | vartime          BN254_Snarks         390.719 ops/s       2559386.7 ns/op (avg)    ± 1.9% (coef var)     633 iterations of      3 operations
𝔾₁ MSM      8192,  16 threads   | vartime          BN254_Snarks         226.191 ops/s       4421043.4 ns/op (avg)    ± 1.7% (coef var)     551 iterations of      2 operations
𝔾₁ MSM     16384,  16 threads   | vartime          BN254_Snarks         118.221 ops/s       8458720.5 ns/op (avg)    ± 2.7% (coef var)     579 iterations of      1 operations
𝔾₁ MSM     32768,  16 threads   | vartime          BN254_Snarks          62.633 ops/s      15965900.6 ns/op (avg)    ± 5.1% (coef var)     306 iterations of      1 operations
𝔾₁ MSM     65536,  16 threads   | vartime          BN254_Snarks          35.109 ops/s      28482414.6 ns/op (avg)    ± 5.6% (coef var)     171 iterations of      1 operations
𝔾₁ MSM    131072,  16 threads   | vartime          BN254_Snarks          19.700 ops/s      50760725.8 ns/op (avg)    ± 2.7% (coef var)      95 iterations of      1 operations
𝔾₁ MSM    262144,  16 threads   | vartime          BN254_Snarks          10.451 ops/s      95680066.8 ns/op (avg)    ± 1.4% (coef var)      48 iterations of      1 operations
𝔾₁ MSM    524288,  16 threads   | vartime          BN254_Snarks           5.388 ops/s     185600865.6 ns/op (avg)    ± 0.9% (coef var)      26 iterations of      1 operations
𝔾₁ MSM   1048576,  16 threads   | vartime          BN254_Snarks           2.773 ops/s     360588126.8 ns/op (avg)    ± 0.3% (coef var)      13 iterations of      1 operations
𝔾₁ MSM   2097152,  16 threads   | vartime          BN254_Snarks           1.402 ops/s     713092947.0 ns/op (avg)    ± 0.1% (coef var)       6 iterations of      1 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₁ Subgroup Check                                  BN254_Snarks             inf ops/s             0.0 ns/op (avg)    ±-nan% (coef var)     500 iterations of 539581 operations
𝔾₁ Hash-to-Curve                                   BN254_Snarks       58781.663 ops/s         17012.1 ns/op (avg)    ± 0.3% (coef var)     500 iterations of    502 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₂ Addition                     | vartime          BN254_Snarks     4294167.661 ops/s           232.9 ns/op (avg)    ± 1.3% (coef var)     500 iterations of  38467 operations
𝔾₂ Scalar Multiplication        | vartime          BN254_Snarks       19703.176 ops/s         50753.2 ns/op (avg)    ± 0.4% (coef var)     501 iterations of    187 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating 2097152 (coefs, points) pairs ... in 13903.610 ms
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₂ MSM         2,  16 threads   | vartime          BN254_Snarks       17023.651 ops/s         58741.8 ns/op (avg)    ± 1.2% (coef var)     502 iterations of    156 operations
𝔾₂ MSM         4,  16 threads   | vartime          BN254_Snarks       15025.275 ops/s         66554.5 ns/op (avg)    ± 1.2% (coef var)     500 iterations of    151 operations
𝔾₂ MSM         8,  16 threads   | vartime          BN254_Snarks       10669.005 ops/s         93729.5 ns/op (avg)    ± 0.6% (coef var)     502 iterations of    104 operations
𝔾₂ MSM        16,  16 threads   | vartime          BN254_Snarks        8778.621 ops/s        113913.1 ns/op (avg)    ± 1.5% (coef var)     500 iterations of     86 operations
𝔾₂ MSM        32,  16 threads   | vartime          BN254_Snarks        5592.212 ops/s        178820.1 ns/op (avg)    ± 0.8% (coef var)     504 iterations of     54 operations
𝔾₂ MSM        64,  16 threads   | vartime          BN254_Snarks        3362.295 ops/s        297415.9 ns/op (avg)    ± 1.9% (coef var)     513 iterations of     32 operations
𝔾₂ MSM       128,  16 threads   | vartime          BN254_Snarks        1815.871 ops/s        550700.0 ns/op (avg)    ± 4.7% (coef var)     519 iterations of     17 operations
𝔾₂ MSM       256,  16 threads   | vartime          BN254_Snarks        1001.467 ops/s        998534.7 ns/op (avg)    ± 2.7% (coef var)     547 iterations of      9 operations
𝔾₂ MSM       512,  16 threads   | vartime          BN254_Snarks         620.579 ops/s       1611399.6 ns/op (avg)    ± 3.0% (coef var)     524 iterations of      6 operations
𝔾₂ MSM      1024,  16 threads   | vartime          BN254_Snarks         368.044 ops/s       2717067.3 ns/op (avg)    ± 3.7% (coef var)     596 iterations of      3 operations
𝔾₂ MSM      2048,  16 threads   | vartime          BN254_Snarks         226.321 ops/s       4418497.4 ns/op (avg)    ± 3.6% (coef var)     547 iterations of      2 operations
𝔾₂ MSM      4096,  16 threads   | vartime          BN254_Snarks         115.509 ops/s       8657366.8 ns/op (avg)    ± 4.9% (coef var)     555 iterations of      1 operations
𝔾₂ MSM      8192,  16 threads   | vartime          BN254_Snarks          84.714 ops/s      11804418.5 ns/op (avg)    ± 2.7% (coef var)     414 iterations of      1 operations
𝔾₂ MSM     16384,  16 threads   | vartime          BN254_Snarks          44.931 ops/s      22256225.6 ns/op (avg)    ± 4.2% (coef var)     216 iterations of      1 operations
𝔾₂ MSM     32768,  16 threads   | vartime          BN254_Snarks          24.989 ops/s      40017277.9 ns/op (avg)    ± 7.1% (coef var)     122 iterations of      1 operations
𝔾₂ MSM     65536,  16 threads   | vartime          BN254_Snarks          13.976 ops/s      71552628.6 ns/op (avg)    ± 5.6% (coef var)      70 iterations of      1 operations
𝔾₂ MSM    131072,  16 threads   | vartime          BN254_Snarks           7.498 ops/s     133370804.3 ns/op (avg)    ± 2.5% (coef var)      34 iterations of      1 operations
𝔾₂ MSM    262144,  16 threads   | vartime          BN254_Snarks           3.970 ops/s     251890430.2 ns/op (avg)    ± 1.2% (coef var)      19 iterations of      1 operations
𝔾₂ MSM    524288,  16 threads   | vartime          BN254_Snarks           2.004 ops/s     498880965.8 ns/op (avg)    ± 0.2% (coef var)       9 iterations of      1 operations
𝔾₂ MSM   1048576,  16 threads   | vartime          BN254_Snarks           1.026 ops/s     974725249.6 ns/op (avg)    ± 0.2% (coef var)       5 iterations of      1 operations
𝔾₂ MSM   2097152,  16 threads   | vartime          BN254_Snarks           0.518 ops/s    1929669910.0 ns/op (avg)    ± 0.2% (coef var)       2 iterations of      1 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
𝔾₂ Subgroup Check                                  BN254_Snarks       15389.808 ops/s         64978.1 ns/op (avg)    ± 1.7% (coef var)     501 iterations of    145 operations
𝔾₂ Hash-to-Curve                                   BN254_Snarks       16363.477 ops/s         61111.7 ns/op (avg)    ± 2.4% (coef var)     500 iterations of    155 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Pairing                                            BN254_Snarks        3322.115 ops/s        301013.0 ns/op (avg)    ± 1.7% (coef var)     501 iterations of     31 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating 1024 (𝔾₁, 𝔾₂) pairs ... in 74.539 ms
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Multipairing     2                                 BN254_Snarks        2538.434 ops/s        393943.7 ns/op (avg)    ± 1.5% (coef var)     513 iterations of     21 operations
Multipairing     4                                 BN254_Snarks        1729.406 ops/s        578233.3 ns/op (avg)    ± 1.3% (coef var)     510 iterations of     16 operations
Multipairing     8                                 BN254_Snarks        1051.776 ops/s        950773.1 ns/op (avg)    ± 1.7% (coef var)     497 iterations of     10 operations
Multipairing    16                                 BN254_Snarks         590.170 ops/s       1694425.9 ns/op (avg)    ± 1.8% (coef var)     557 iterations of      5 operations
Multipairing    32                                 BN254_Snarks         315.046 ops/s       3174140.9 ns/op (avg)    ± 1.3% (coef var)     497 iterations of      3 operations
Multipairing    64                                 BN254_Snarks         162.836 ops/s       6141136.0 ns/op (avg)    ± 2.1% (coef var)     758 iterations of      1 operations
Multipairing   128                                 BN254_Snarks          82.578 ops/s      12109777.2 ns/op (avg)    ± 1.6% (coef var)     384 iterations of      1 operations
Multipairing   256                                 BN254_Snarks          41.852 ops/s      23893775.6 ns/op (avg)    ± 1.3% (coef var)     198 iterations of      1 operations
Multipairing   512                                 BN254_Snarks          21.028 ops/s      47556249.4 ns/op (avg)    ± 0.9% (coef var)      99 iterations of      1 operations
Multipairing  1024                                 BN254_Snarks          10.553 ops/s      94763650.4 ns/op (avg)    ± 0.7% (coef var)      49 iterations of      1 operations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

json

{"add_ff":{"range":[1],"results":[1.0],"stddev":[0.0]},"mul_ff":{"range":[1],"results":[14.0],"stddev":[0.0]},"invert":{"range":[1],"results":[761.3640000000001],"stddev":[2.221119254223629]},"ip_ff":{"range":[2],"results":[17.978],"stddev":[0.1468302361112071]},"fft":{"range":[],"results":[],"stddev":[]},"add_G1":{"range":[1],"results":[94.53199999999991],"stddev":[0.5527993979272262]},"mul_G1":{"range":[1],"results":[26850.41000000001],"stddev":[86.65168740341818]},"msm_G1":{"range":[2,4,8,16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,131072,262144,524288,1048576,2097152],"results":[44910.58600000001,43585.76199999998,51472.56200000001,62400.23153692614,86233.85458167328,128594.6719681908,211492.5436507938,368969.6706114396,579930.2450199202,940740.7601476017,1525638.799628944,2559386.734597154,4421043.353901998,8458720.5060449,15965900.63398693,28482414.58479532,50760725.78947367,95680066.83333333,185600865.576923,360588126.8461539,713092947.0],"stddev":[887.0271794246637,276.9817668264606,549.3243601564757,449.5514678915364,499.6471976551839,596.3731098940373,2390.240447673251,5847.154740662249,4405.894348676709,12383.13910067745,27909.37999982535,48203.61076122781,75430.42635452512,232201.5774550728,817882.6957315446,1598699.54483082,1345263.885652233,1353439.608259127,1716016.148141342,1188959.211887081,581084.7686257195]},"is_in_sub_G1":{"range":[1],"results":[0.0],"stddev":[0.0]},"hash_G1":{"range":[1],"results":[17012.10800000001],"stddev":[57.10332666168426]},"add_G2":{"range":[1],"results":[232.8740000000001],"stddev":[2.968458774345699]},"mul_G2":{"range":[1],"results":[50753.23952095804],"stddev":[202.8829182434307]},"msm_G2":{"range":[2,4,8,16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,131072,262144,524288,1048576,2097152],"results":[58741.80677290835,66554.52399999993,93729.45219123513,113913.102,178820.117063492,297415.8849902534,550699.9980732185,998534.7221206586,1611399.599236641,2717067.295302014,4418497.376599634,8657366.819819819,11804418.52415459,22256225.56018518,40017277.94262295,71552628.61428569,133370804.2941176,251890430.2105263,498880965.7777778,974725249.6,1929669910.0],"stddev":[696.1802888080764,771.7982135174744,603.7003466026882,1716.632938308528,1363.800684245473,5604.649784897922,25820.10754155615,26624.50358125539,47964.9816976701,100746.5209993159,158918.1137742022,424028.0461159708,320727.7412293794,944649.8402733286,2855466.92962508,3994160.162350956,3280845.204295911,2918838.609642661,954034.4469252342,2288634.168998681,3284550.59659126]},"is_in_sub_G2":{"range":[1],"results":[64978.06786427147],"stddev":[1123.100064724973]},"hash_G2":{"range":[1],"results":[61111.70600000001],"stddev":[1453.380319210541]},"mul_Gt":{"range":[],"results":[],"stddev":[]},"multiexp_Gt":{"range":[],"results":[],"stddev":[]},"pairing":{"range":[1],"results":[301013.0199600802],"stddev":[5116.641067302012]},"multipairing":{"range":[2,4,8,16,32,64,128,256,512,1024],"results":[393943.6764132551,578233.2568627453,950773.092555332,1694425.947935367,3174140.85915493,6141135.969656986,12109777.24739584,23893775.60101011,47556249.37373736,94763650.36734694],"stddev":[5764.176315676999,7396.070709490642,16358.28938354904,30757.00252728,40519.90862940051,130609.6644831436,196746.4353761875,299893.8500157776,423738.3472848108,671637.7269323865]}}

@mratsim mratsim marked this pull request as ready for review July 12, 2024 20:39
@mratsim mratsim merged commit 52a8fc4 into master Jul 12, 2024
24 checks passed
@mratsim mratsim deleted the zkalc branch July 12, 2024 23:28
mratsim added a commit that referenced this pull request Jul 15, 2024
mratsim added a commit that referenced this pull request Jul 15, 2024
* feat(𝔾ₜ constant-time exponentiation): generalize ec_endomorphism_accel to elliptic curve and 𝔾ₜ

* feat(𝔾ₜ constant-time exponentiation): implement constant-time 𝔾ₜ exponentiation

* feat(gt-exp): public API + zkalc bench #425
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant