You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, could you please help with resolving below issue:
By running the simulator with all the corresponding parameters, I can obtain the predicting execution time for different models. But the output always like this:
......
INFO 10-16 22:30:40 sklearn_execution_time_predictor.py:433] Predicting execution time for model attn_decode
INFO 10-16 22:30:41 simulator.py:60] Starting simulation with cluster: Cluster({'id': 0, 'num_replicas': 4}) and 128 requests
INFO 10-16 22:30:43 simulator.py:80] Simulation ended at: 41.85942038841162s
INFO 10-16 22:30:44 simulator.py:83] Writing output
INFO 10-16 22:30:47 simulator.py:86] Metrics written
INFO 10-16 22:30:47 simulator.py:94] Chrome event trace written
The id seems to increase, does this mean that, the simulator supports multiple clusters, as contemporaneous input?How to use it?
Go further, whether to support multiple LLM inputs?
The text was updated successfully, but these errors were encountered:
Hi, could you please help with resolving below issue:
By running the simulator with all the corresponding parameters, I can obtain the predicting execution time for different models. But the output always like this:
......
INFO 10-16 22:30:40 sklearn_execution_time_predictor.py:433] Predicting execution time for model attn_decode
INFO 10-16 22:30:41 simulator.py:60] Starting simulation with cluster: Cluster({'id': 0, 'num_replicas': 4}) and 128 requests
INFO 10-16 22:30:43 simulator.py:80] Simulation ended at: 41.85942038841162s
INFO 10-16 22:30:44 simulator.py:83] Writing output
INFO 10-16 22:30:47 simulator.py:86] Metrics written
INFO 10-16 22:30:47 simulator.py:94] Chrome event trace written
The id of the Cluster is always 0, but :
The id seems to increase, does this mean that, the simulator supports multiple clusters, as contemporaneous input?How to use it?
Go further, whether to support multiple LLM inputs?
The text was updated successfully, but these errors were encountered: