Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a large amount of dev data #16925

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

ncounter
Copy link
Contributor

@ncounter ncounter commented Oct 7, 2024

An already set up database must exists ahead (rake dev:test_data:create)

Run:

  • bundle exec rake dev:test_data:create_more_data[100] if inside the container
  • docker compose run --rm frontend rake dev:test_data:create_more_data[100] if outside of the container

where [100] can be any number you desire to flood your db with (the number represents the repetitions the create objects tasks are executed).

@github-actions github-actions bot added the Frontend Things related to the OBS RoR app label Oct 7, 2024
@ncounter ncounter changed the title Create a large amount of data to test Create a large amount of dev data Oct 7, 2024
@ncounter ncounter force-pushed the test-data-at-scale branch 2 times, most recently from 5c135f7 to 4c58c48 Compare October 7, 2024 15:01
@danidoni
Copy link
Contributor

danidoni commented Oct 8, 2024

Why don't we use this as the default data creation?

@ncounter
Copy link
Contributor Author

ncounter commented Oct 8, 2024

Why don't we use this as the default data creation?

That's a valid point to discuss further: initially my attempt was meant to be a follow-up step, just in case there is a need to test the app performance wise, without affecting the existing setup. Basically I see some points to adjust if we want to squash the create and create_more_data into a default one task:

  • it may take a lot of time depending on the repetitions value
    • what to do with the special corner cases like Rake::Task['dev:requests:request_with_multiple_submit_actions_builds_and_diffs'].invoke(1, 1000), it's not a default case we want to set up all the time, but it may be useful to create on purpose
  • how to handle the initialization of the database and all the sub steps that are not meant to be repeated
  • at the moment the default create destroys and sets up a new db on every call; the create_more_data is something like and idempotent task instead, it always does the same change without changing the existing data

Ideally, we should split better what is an initial db setup that is required in order to play with the app (the minimal data), and what is a pure data_creation that can be called as much as we want to increase the db size.

@danidoni
Copy link
Contributor

danidoni commented Oct 8, 2024

Why don't we use this as the default data creation?

That's a valid point to discuss further: initially my attempt was meant to be a follow-up step, just in case there is a need to test the app performance wise, without affecting the existing setup. Basically I see some points to adjust if we want to squash the create and create_more_data into a default one task:

* it may take a lot of time depending on the repetitions value
  
  * what to do with the special corner cases like `Rake::Task['dev:requests:request_with_multiple_submit_actions_builds_and_diffs'].invoke(1, 1000)`, it's not a default case we want to set up all the time, but it may be useful to create on purpose

* how to handle the initialization of the database and all the sub steps that are not meant to be repeated

* at the moment the default `create` destroys and sets up a new db on every call; the `create_more_data` is something like and idempotent task instead, it always does the same change without changing the existing data

Ideally, we should split better what is an initial db setup that is required in order to play with the app (the minimal data), and what is a pure data_creation that can be called as much as we want to increase the db size.

I see

@ncounter ncounter marked this pull request as ready for review October 9, 2024 13:19
target_package: target_package_b
}
bs_req_action = build(:bs_request_action, action_attributes.merge(type: 'submit', bs_request: request))
bs_req_action.save! if bs_req_action.valid?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not valid, what happens with this action?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it doesn't get saved thus it doesn't get into the request that has being created. It's the previous already existing logic implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why it cannot be valid?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Frontend Things related to the OBS RoR app
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants