Fix/put on busy shards #2965
Fix/put on busy shards #2965
Conversation
carpawell
force-pushed
the
fix/put-on-busy-shards
branch
from
0f8b19e
to
03f1f87
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2965 +/- ##
==========================================
- Coverage 23.47% 23.47% -0.01%
==========================================
Files 780 780
Lines 46655 46688 +33
==========================================
+ Hits 10953 10958 +5
- Misses 34833 34859 +26
- Partials 869 871 +2 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Pavel Karpy <[email protected]>
carpawell
force-pushed
the
fix/put-on-busy-shards
branch
from
03f1f87
to
13b546f
Compare
If every shard's pool is overloaded with routines, choose the best one and try to PUT an object to it 30 seconds. Closes #2871. Signed-off-by: Pavel Karpy <[email protected]>
carpawell
force-pushed
the
fix/put-on-busy-shards
branch
from
13b546f
to
12246c2
Compare
carpawell
requested review from
roman-khimov and
cthulhu-rider
as code owners
|
Well, it works. There is not much we can do about testing currently. It is hard to test in general because it is time-dependent. But it is quite straightforward code, and the tests are okay. |
| } | ||
|
|
||
| func (e *StorageEngine) putToShardWithDeadLine(sh hashedShard, ind int, pool util.WorkerPool, addr oid.Address, prm PutPrm) error { | ||
| const deadline = 30 * time.Second |
There was a problem hiding this comment.
Make it configurable? Treat zero as old behavior?
| err = errPutShard | ||
| err = e.putToShardWithDeadLine(bestShard, 0, bestPool, addr, prm) | ||
| if err != nil { | ||
| e.log.Warn("last stand to put object to the best shard", |
There was a problem hiding this comment.
Doesn't deserve a warn to me.
| @@ -167,19 +214,20 @@ func (e *StorageEngine) putToShard(sh hashedShard, ind int, pool util.WorkerPool | |||
| return | |||
| } | |||
|
|
|||
| e.reportShardError(sh, "could not put object to shard", err) | |||
| e.reportShardError(sh, "could not put object to shard", errGlobal) | |||
| defer timer.Stop() | ||
|
|
||
| const putCooldown = 100 * time.Millisecond | ||
| ticker := time.NewTicker(putCooldown) |
There was a problem hiding this comment.
One of the reasons I don't like pools we have now. Channels provide natural queueing and blocking/unblocking.
@roman-khimov, what do you mean by that? Sending tasks via channels to every shard? What is the difference? |
|
How can I get a "block for 30s" behavior with a pool? How can I get a queue with a pool? That's the difference. |
|
|
No description provided.