Unable to get Encore Requests Per Second to be the Highest

[jefflam]

Hey Encore team,

Just discovered Encore and really love the vision and the promise for performance by leveraging on Rust's run-time.

However, I'm struggling to actually get the highest RPS using oha running on my local machine (M2 Pro Macbook Pro) against the requests/encore local server, ran via encore run.

Just comparing against encore vs elysium for non schema validation API calls (GETs), I'm getting...

Elysium

 ❯ oha -c 150 -z 10s -m GET  -H 'Content-Type: application/json' -H 'x-foo: test' "http://127.0.0.1:3000/hello"
Summary:
  Success rate: 100.00%
  Total:        10.0029 secs
  Slowest:      0.1825 secs
  Fastest:      0.0001 secs
  Average:      0.0013 secs
  Requests/sec: 111210.4259

  Total data:   27.58 MiB
  Size/request: 26 B
  Size/sec:     2.76 MiB

Response time histogram:
  0.000 [1]       |
  0.018 [1112249] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.037 [129]     |
  0.055 [0]       |
  0.073 [0]       |
  0.091 [0]       |
  0.110 [0]       |
  0.128 [0]       |
  0.146 [0]       |
  0.164 [0]       |
  0.182 [21]      |

Response time distribution:
  10.00% in 0.0012 secs
  25.00% in 0.0013 secs
  50.00% in 0.0013 secs
  75.00% in 0.0013 secs
  90.00% in 0.0016 secs
  95.00% in 0.0017 secs
  99.00% in 0.0024 secs
  99.90% in 0.0062 secs
  99.99% in 0.0210 secs


Details (average, fastest, slowest):
  DNS+dialup:   0.0309 secs, 0.0036 secs, 0.1813 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0005 secs

Status code distribution:
  [200] 1112400 responses

Error distribution:
  [21] connection error
  [5] aborted due to deadline

Encore

 ❯ oha -c 150 -z 10s -m GET  -H 'Content-Type: application/json' -H 'x-foo: test' "http://127.0.0.1:4000/hello" 
Summary:
  Success rate: 99.93%
  Total:        10.0216 secs
  Slowest:      0.7172 secs
  Fastest:      0.0018 secs
  Average:      0.0585 secs
  Requests/sec: 2557.1762

  Total data:   616.64 KiB
  Size/request: 24 B
  Size/sec:     61.53 KiB

Response time histogram:
  0.002 [1]     |
  0.073 [19851] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.145 [3968]  |■■■■■■
  0.216 [734]   |■
  0.288 [492]   |
  0.360 [142]   |
  0.431 [75]    |
  0.503 [75]    |
  0.574 [62]    |
  0.646 [53]    |
  0.717 [7]     |

Response time distribution:
  10.00% in 0.0132 secs
  25.00% in 0.0198 secs
  50.00% in 0.0420 secs
  75.00% in 0.0683 secs
  90.00% in 0.1124 secs
  95.00% in 0.1684 secs
  99.00% in 0.3676 secs
  99.90% in 0.6296 secs
  99.99% in 0.7155 secs


Details (average, fastest, slowest):
  DNS+dialup:   0.0063 secs, 0.0018 secs, 0.0082 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0007 secs

Status code distribution:
  [200] 24286 responses
  [502] 1174 responses

Error distribution:
  [150] aborted due to deadline
  [17] connection error

I've ran this test a few times all on my local machine again and consistently get these results of 111210 requests vs 2557 requests..a 43x higher RPS for Elysia.

This goes complete against what you guys shared; but at such a large difference, I do feel like I'm missing something.

Writing this here to see if you guys have any insights?

Thanks!