Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update result docs #44

Open
jamadden opened this issue May 31, 2019 · 1 comment
Open

Update result docs #44

jamadden opened this issue May 31, 2019 · 1 comment

Comments

@jamadden
Copy link
Member

The page https://zodbshootout.readthedocs.io/en/latest/results.html is a bit outdated with respect to 0.7.0 and its new measurements.

@jamadden
Copy link
Member Author

jamadden commented Jun 1, 2019

Python 3.7, concurrency = 5 processes (zeo is not using a persistent cache, despite the name, because that doesn't work with mulitple processes):

MappingStorage: add 1000 objects: Mean +- std dev: 54.2 ms +- 3.2 ms
MappingStorage: update 1000 objects: Mean +- std dev: 51.8 ms +- 1.9 ms
MappingStorage: read 1000 cold objects: Mean +- std dev: 36.4 ms +- 1.7 ms
MappingStorage: read 1000 warm objects: Mean +- std dev: 87.9 ms +- 8.0 ms
MappingStorage: read 1000 hot objects: Mean +- std dev: 43.0 ms +- 2.3 ms
MappingStorage: read 1000 steamin objects: Mean +- std dev: 1.12 ms +- 0.19 ms
MappingStorage: empty commit: Mean +- std dev: 42.3 us +- 2.4 us
zeofs_pcache: add 1000 objects: Mean +- std dev: 528 ms +- 16 ms
zeofs_pcache: update 1000 objects: Mean +- std dev: 636 ms +- 7 ms
zeofs_pcache: read 1000 cold objects: Mean +- std dev: 764 ms +- 9 ms
zeofs_pcache: read 1000 warm objects: Mean +- std dev: 1.39 sec +- 0.04 sec
zeofs_pcache: read 1000 hot objects: Mean +- std dev: 50.7 ms +- 2.4 ms
zeofs_pcache: read 1000 steamin objects: Mean +- std dev: 1.15 ms +- 0.12 ms
zeofs_pcache: empty commit: Mean +- std dev: 38.4 us +- 1.5 us
psycopg2-pcache: add 1000 objects: Mean +- std dev: 155 ms +- 3 ms
psycopg2-pcache: update 1000 objects: Mean +- std dev: 145 ms +- 2 ms
psycopg2-pcache: read 1000 cold objects: Mean +- std dev: 73.8 ms +- 3.4 ms
psycopg2-pcache: read 1000 warm objects: Mean +- std dev: 256 ms +- 13 ms
psycopg2-pcache: read 1000 hot objects: Mean +- std dev: 65.9 ms +- 4.0 ms
psycopg2-pcache: read 1000 steamin objects: Mean +- std dev: 1.05 ms +- 0.06 ms
psycopg2-pcache: empty commit: Mean +- std dev: 364 us +- 10 us
mysqlclient-pcache: add 1000 objects: Mean +- std dev: 157 ms +- 2 ms
mysqlclient-pcache: update 1000 objects: Mean +- std dev: 215 ms +- 4 ms
mysqlclient-pcache: read 1000 cold objects: Mean +- std dev: 76.3 ms +- 3.0 ms
mysqlclient-pcache: read 1000 warm objects: Mean +- std dev: 338 ms +- 18 ms
mysqlclient-pcache: read 1000 hot objects: Mean +- std dev: 62.4 ms +- 4.0 ms
mysqlclient-pcache: read 1000 steamin objects: Mean +- std dev: 1.02 ms +- 0.06 ms
mysqlclient-pcache: empty commit: Mean +- std dev: 528 us +- 10 us
mysqlclient-hp: add 1000 objects: Mean +- std dev: 176 ms +- 4 ms
mysqlclient-hp: update 1000 objects: Mean +- std dev: 236 ms +- 3 ms
mysqlclient-hp: read 1000 cold objects: Mean +- std dev: 481 ms +- 173 ms
mysqlclient-hp: read 1000 warm objects: Mean +- std dev: 703 ms +- 28 ms
mysqlclient-hp: read 1000 hot objects: Mean +- std dev: 59.1 ms +- 2.5 ms
mysqlclient-hp: read 1000 steamin objects: Mean +- std dev: 1.03 ms +- 0.04 ms
mysqlclient-hp: empty commit: Mean +- std dev: 573 us +- 31 us
Benchmark zeofs_pcache_1000 mysqlclient-hp_1000 mysqlclient-pcache_1000 psycopg2-pcache_1000
add 528 ms 176 ms: 3.00x faster (-67%) 157 ms: 3.37x faster (-70%) 155 ms: 3.39x faster (-71%)
update 636 ms 236 ms: 2.69x faster (-63%) 215 ms: 2.95x faster (-66%) 145 ms: 4.38x faster (-77%)
cold 764 ms 481 ms: 1.59x faster (-37%) 76.3 ms: 10.02x faster (-90%) 73.8 ms: 10.36x faster (-90%)
warm 1.39 sec 703 ms: 1.97x faster (-49%) 338 ms: 4.10x faster (-76%) 256 ms: 5.42x faster (-82%)
hot 50.7 ms 59.1 ms: 1.17x slower (+17%) 62.4 ms: 1.23x slower (+23%) 65.9 ms: 1.30x slower (+30%)
steamin 1.15 ms 1.03 ms: 1.12x faster (-11%) 1.02 ms: 1.12x faster (-11%) not significant
commit 38.4 us 573 us: 14.89x slower (+1389%) 528 us: 13.75x slower (+1275%) 364 us: 9.48x slower (+848%)

EDIT: I should mention that those incorporate the persistent caching changes from zodb/relstorage#243

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant