-
Notifications
You must be signed in to change notification settings - Fork 294
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add the benchmarking config used in the announcement post on swift.org
- Loading branch information
Showing
12 changed files
with
226 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
{ | ||
"kind": "group", | ||
"title": "All results", | ||
"directory": "Results", | ||
"contents": [ | ||
{ | ||
"kind": "chart", | ||
"title": "Deque: random access lookups", | ||
"tasks": [ | ||
"Deque<Int> subscript get, random offsets (discontiguous)", | ||
"Array<Int> subscript get, random offsets", | ||
"std::deque<intptr_t> at, random offsets", | ||
] | ||
}, | ||
{ | ||
"kind": "chart", | ||
"title": "Deque: prepending individual integers", | ||
"tasks": [ | ||
"Deque<Int> prepend, reserving capacity", | ||
"Array<Int> prepend, reserving capacity", | ||
"std::deque<intptr_t> push_front", | ||
] | ||
}, | ||
{ | ||
"kind": "chart", | ||
"title": "OrderedSet lookups", | ||
"tasks": [ | ||
"OrderedSet<Int> successful contains", | ||
"Set<Int> successful contains", | ||
"std::unordered_set<intptr_t> successful find", | ||
"Array<Int> successful contains" | ||
] | ||
}, | ||
{ | ||
"kind": "chart", | ||
"title": "OrderedSet insertions", | ||
"tasks": [ | ||
"OrderedSet<Int> append, reserving capacity", | ||
"Set<Int> insert, reserving capacity", | ||
"std::unordered_set<intptr_t> insert, reserving capacity", | ||
] | ||
}, | ||
{ | ||
"kind": "chart", | ||
"title": "OrderedDictionary lookups", | ||
"tasks": [ | ||
"OrderedDictionary<Int, Int> subscript, successful lookups", | ||
"Dictionary<Int, Int> subscript, successful lookups", | ||
"std::unordered_map<intptr_t, intptr_t> successful find", | ||
] | ||
}, | ||
{ | ||
"kind": "chart", | ||
"title": "OrderedDictionary insertions", | ||
"tasks": [ | ||
"OrderedDictionary<Int, Int> subscript, append, reserving capacity", | ||
"Dictionary<Int, Int> subscript, insert, reserving capacity", | ||
"std::unordered_map<intptr_t, intptr_t> insert, reserving capacity", | ||
] | ||
}, | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Benchmark data from the swift.org announcement | ||
|
||
This directory contains the benchmark configuration that we used to collect benchmarking data and render the charts in the [Swift Collections announcement on swift.org][announcement]. | ||
|
||
* [`generate-results.sh`](./generate-results.sh): A script that you can use to run the benchmark on your own machine, and to reproduce the charts. | ||
* [`Library.json`](./Library.json): The benchmark library definition collecting the chart definitions used in the blog post. | ||
* [`Theme.json`](./Theme.json): The chart theme to set up colors, line widths, font sizes etc. (Slightly adapted.) | ||
* [`Results.md`](./Results.md): The generated results summary. | ||
* [`Results/`](./Results/): The generated subdirectory containing the PNG files for the charts. | ||
* [`results.json`](./results.json): The result data we collected. | ||
|
||
Note: If you'd like to try reproducing our results (it's easy!), note that the script is configured to collect 20 rounds of data on up to 16 million items, and this will likely take a very long time. Feel free to reduce the maximum size, or just plan to allow the measurement run overnight. | ||
|
||
For more information on how to use the Swift Collections benchmarking tool, please see [Swift Collections Benchmark][swift-collections-benchmark]. | ||
|
||
[announcement]: https://swift.org/blog/swift-collections | ||
[swift-collections-benchmark]: https://github.com/apple/swift-collections-benchmark |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Benchmark results | ||
|
||
Click to expand individual items below. | ||
<details open> | ||
<summary><strong>All results</strong></summary> | ||
<ul> | ||
<details> | ||
<summary>01: Deque: random access lookups</summary> | ||
<img src="Results/01%20Deque:%20random%20access%20lookups.png"> | ||
</details> | ||
<details> | ||
<summary>02: Deque: prepending individual integers</summary> | ||
<img src="Results/02%20Deque:%20prepending%20individual%20integers.png"> | ||
</details> | ||
<details> | ||
<summary>03: OrderedSet lookups</summary> | ||
<img src="Results/03%20OrderedSet%20lookups.png"> | ||
</details> | ||
<details> | ||
<summary>04: OrderedSet insertions</summary> | ||
<img src="Results/04%20OrderedSet%20insertions.png"> | ||
</details> | ||
<details> | ||
<summary>05: OrderedDictionary lookups</summary> | ||
<img src="Results/05%20OrderedDictionary%20lookups.png"> | ||
</details> | ||
<details> | ||
<summary>06: OrderedDictionary insertions</summary> | ||
<img src="Results/06%20OrderedDictionary%20insertions.png"> | ||
</details> | ||
</ul> | ||
</details> |
Binary file added
BIN
+342 KB
Documentation/Announcement-benchmarks/Results/01 Deque: random access lookups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+434 KB
...on/Announcement-benchmarks/Results/02 Deque: prepending individual integers.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+565 KB
Documentation/Announcement-benchmarks/Results/03 OrderedSet lookups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+472 KB
Documentation/Announcement-benchmarks/Results/04 OrderedSet insertions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+468 KB
Documentation/Announcement-benchmarks/Results/05 OrderedDictionary lookups.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+468 KB
Documentation/Announcement-benchmarks/Results/06 OrderedDictionary insertions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
{ | ||
"axisLabels" : { | ||
"color" : "#000000FF", | ||
"font" : { | ||
"family" : "Helvetica", | ||
"isBold" : false, | ||
"isItalic" : false, | ||
"size" : 13 | ||
} | ||
}, | ||
"axisLeading" : 3, | ||
"background" : "#FFFFFFFF", | ||
"bandDimmingFactor" : 0.29999999999999999, | ||
"border" : { | ||
"capStyle" : "round", | ||
"color" : "#000000FF", | ||
"joinStyle" : "round", | ||
"width" : 0.5 | ||
}, | ||
"curveFallback" : { | ||
"color" : "#000000FF", | ||
"lineWidth" : 4 | ||
}, | ||
"curves" : [ | ||
{ | ||
"color" : "#F05138FF", | ||
"lineWidth" : 8 | ||
}, | ||
{ | ||
"color" : "#FFC07FFF", | ||
"lineWidth" : 6 | ||
}, | ||
{ | ||
"color" : "#FFE57FFF", | ||
"lineWidth" : 6 | ||
}, | ||
{ | ||
"color" : "#93E6C0FF", | ||
"lineWidth" : 6 | ||
}, | ||
], | ||
"hairlines" : { | ||
"capStyle" : "round", | ||
"color" : "#0000004D", | ||
"joinStyle" : "round", | ||
"width" : 0.5 | ||
}, | ||
"legendCornerOffset" : [ | ||
24, | ||
24 | ||
], | ||
"legendLabels" : { | ||
"color" : "#000000FF", | ||
"font" : { | ||
"family" : "Menlo", | ||
"isBold" : false, | ||
"isItalic" : false, | ||
"size" : 13 | ||
} | ||
}, | ||
"legendLineLeading" : 3, | ||
"legendLineSampleWidth" : 24, | ||
"legendPadding" : { | ||
"bottom" : 12, | ||
"left" : 12, | ||
"right" : 12, | ||
"top" : 12 | ||
}, | ||
"legendPosition" : "topLeft", | ||
"legendSeparation" : 9, | ||
"majorGridline" : { | ||
"capStyle" : "butt", | ||
"color" : "#0000007F", | ||
"joinStyle" : "round", | ||
"width" : 0.75 | ||
}, | ||
"margins" : { | ||
"bottom" : 9, | ||
"left" : 9, | ||
"right" : 9, | ||
"top" : 9 | ||
}, | ||
"minorGridline" : { | ||
"capStyle" : "butt", | ||
"color" : "#00000042", | ||
"dash" : { | ||
"phase" : 0, | ||
"style" : [ | ||
6, | ||
3 | ||
] | ||
}, | ||
"joinStyle" : "round", | ||
"width" : 0.5 | ||
}, | ||
"xPadding" : 6, | ||
"yPadding" : 3 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
#!/bin/sh | ||
#===----------------------------------------------------------------------===// | ||
# | ||
# This source file is part of the Swift Collections open source project | ||
# | ||
# Copyright (c) 2021 Apple Inc. and the Swift project authors | ||
# Licensed under Apache License v2.0 with Runtime Library Exception | ||
# | ||
# See https://swift.org/LICENSE.txt for license information | ||
# | ||
#===----------------------------------------------------------------------===// | ||
|
||
set -eux | ||
|
||
../../Utils/run-benchmarks.sh library run results.json --library Library.json --max-size 16M --cycles 20 --mode replace-all | ||
../../Utils/run-benchmarks.sh library render results.json --library Library.json --max-time 10us --min-time 1ns --theme-file Theme.json --percentile 90 --output . |
Large diffs are not rendered by default.
Oops, something went wrong.