Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can this simulator supports multiple clusters or multiple LLM inputs? #41

Open
GRYYYYYYY opened this issue Oct 16, 2024 · 0 comments
Open

Comments

@GRYYYYYYY
Copy link

Hi, could you please help with resolving below issue:

By running the simulator with all the corresponding parameters, I can obtain the predicting execution time for different models. But the output always like this:
......
INFO 10-16 22:30:40 sklearn_execution_time_predictor.py:433] Predicting execution time for model attn_decode
INFO 10-16 22:30:41 simulator.py:60] Starting simulation with cluster: Cluster({'id': 0, 'num_replicas': 4}) and 128 requests
INFO 10-16 22:30:43 simulator.py:80] Simulation ended at: 41.85942038841162s
INFO 10-16 22:30:44 simulator.py:83] Writing output
INFO 10-16 22:30:47 simulator.py:86] Metrics written
INFO 10-16 22:30:47 simulator.py:94] Chrome event trace written

The id of the Cluster is always 0, but :

@classmethod
def generate_id(cls):
    cls._id += 1
    return cls._id

The id seems to increase, does this mean that, the simulator supports multiple clusters, as contemporaneous input?How to use it?
Go further, whether to support multiple LLM inputs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant