Integrate meta ai and other changes from LSC DC demo #163
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
jimmytyyang
commented
Apr 22, 2024
•
jimmytyyang
commented
- demo branch in DC
- meta ai integration
- use skill manager for lsc
facebook-github-bot
added
the
CLA Signed
jimmytyyang
force-pushed
the
update_openai_model_nyc
branch
from
fe72fbc
to
56c2f41
Compare
jimmytyyang
commented
May 3, 2024
KavitShah1998
force-pushed
the
update_openai_model_nyc
branch
from
3e6d58c
to
ef8098c
Compare
KavitShah1998
force-pushed
the
update_openai_model_nyc
branch
from
4f40ec2
to
77f75c8
Compare
…t undock, add safety-net to language parsing
KavitShah1998
force-pushed
the
update_openai_model_nyc
branch
from
77f75c8
to
f9cfc94
Compare
KavitShah1998
changed the title
KavitShah1998
changed the title
tusharsangam
pushed a commit
that referenced
this pull request
Aug 6, 2024
…t undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp
tusharsangam
added a commit
that referenced
this pull request
Aug 6, 2024
* File add for contrained place point estimation * nyc and fre placement * nyc and fre placement * tidy * add support of proposition * generate the image services automatically * move rospy * tidy * tidy --------- Co-authored-by: Jimmy Yang <[email protected]> Integrate meta ai, add llama3 local support, read reset_home params at undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp
tusharsangam
added a commit
that referenced
this pull request
Aug 6, 2024
* File add for contrained place point estimation * nyc and fre placement * nyc and fre placement * tidy * add support of proposition * generate the image services automatically * move rospy * tidy * tidy --------- Co-authored-by: Jimmy Yang <[email protected]> Integrate meta ai, add llama3 local support, read reset_home params at undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp
tusharsangam
added a commit
that referenced
this pull request
Aug 8, 2024
* File add for contrained place point estimation * nyc and fre placement * nyc and fre placement * tidy * add support of proposition * generate the image services automatically * move rospy * tidy * tidy --------- Co-authored-by: Jimmy Yang <[email protected]> Integrate meta ai, add llama3 local support, read reset_home params at undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp rebasing error fix rebase fix pose estimation all testing removing semantic pick testing artifacts
tusharsangam
added a commit
that referenced
this pull request
Aug 8, 2024
* File add for contrained place point estimation * nyc and fre placement * nyc and fre placement * tidy * add support of proposition * generate the image services automatically * move rospy * tidy * tidy --------- Co-authored-by: Jimmy Yang <[email protected]> Integrate meta ai, add llama3 local support, read reset_home params at undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp rebasing error fix rebase fix pose estimation all testing removing semantic pick testing artifacts
tusharsangam
added a commit
that referenced
this pull request
Aug 17, 2024
* load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> Ollama updates (#175) * load the newly trained checkpoints * port semantic pick * tidy * appraching * update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well * smooth arm actions * Chris Cox Demo test2 lsc_demo.py fix keep_alive change parameter update for metaailocal api Reducing time for Meta AI Local & occassional format crashing issues loading whisper on cuda:1 fixing precommit * uncommenting pose estimation * restoring keyboard teleop * deleting test_semantic_gaze.py * polish --------- Co-authored-by: Jimmy Yang <[email protected]> Semantic pick with force control (#176) * load the newly trained checkpoints * port semantic pick * tidy * appraching * update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well * smooth arm actions * first commit * force control first commit * parameter adjustment * Generalization to grasp * some changes remaining * arm command follow * pose estimation fix for grasping * video shoot * body follow fix * some changes * caliberation fix * Resolving calibartaion error with minimal changes * gitignore * backup * benchmark files * ros_communications (#173) reducing file size Bug fixes n support for Float32MultiArray bug fixes after PPG with Jimmy fixing pre-commit precommit fix 2 * gripper_T_intel * adding new objects in config * patch fix for ros tcp * gitignore * cleanup * removing unnecessary * ros tcp fix * ros test update * pose correction + BD grasp, small patch * clean up --------- Co-authored-by: Jimmy Yang <[email protected]> Co-authored-by: Jimmy Yang <[email protected]> Grasp config fix (#180) * load the newly trained checkpoints * port semantic pick * tidy * appraching * update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well * smooth arm actions * grasp mode loading from config * reverting test_pick_with_pose.py * polish --------- Co-authored-by: Jimmy Yang <[email protected]> Co-authored-by: Jimmy Yang <[email protected]> Improve keyboard-teleop and datalogger (#174) File add for constrained place point estimation (#162) * File add for contrained place point estimation * nyc and fre placement * nyc and fre placement * tidy * add support of proposition * generate the image services automatically * move rospy * tidy * tidy --------- Co-authored-by: Jimmy Yang <[email protected]> Integrate meta ai, add llama3 local support, read reset_home params at undock, add safety-net to language parsing (#163) pose estimation (#166) * load the newly trained checkpoints port semantic pick tidy appraching update checkpoint and make the MAX_JOINT_MOVEMENT_SEMANTIC_GAZE large and it works well smooth arm actions pose estimation pose correction with discretization Orientation Correction Working pose correction finetuning adding more grasp orientation & object correction pose estimation with object facing camera creating new pick api force control for object grasping PR ready to pr PR ready changes Starting pose estimation service from spot-sim2real PR hot fixes submodule init instruction update add todo and fix pre-commit issue PR review 1 PR fixes 2 fix precommit fix precommit v2 * add trest srcipt for nn basew * PR review final --------- Co-authored-by: Jimmy Yang <[email protected]> first commit force control first commit parameter adjustment Generalization to grasp some changes remaining arm command follow pose estimation fix for grasping video shoot body follow fix some changes caliberation fix Resolving calibartaion error with minimal changes gitignore saving hanges running episode tests joannes apt changes joanna changes remaining changes running pose estimation correction with BD grasp pose correction + BD grasp force control after BD grasp rebasing error fix rebase fix pose estimation all testing removing semantic pick testing artifacts changes backup 08/08 semantic place + waypoints fix semantic place + waypoints epsidoe backup we don't need fix semantic place working save table location backup Vyshnav changes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.