Skip to content

Commit

Permalink
updated v1.0.0
Browse files Browse the repository at this point in the history
- Gives better parsing of tasking info
- Search by task ID now
- Now outputs EECU and path for tasks that have those fields
- Fixed size estimation for image vs image collection
- Enhanced user docs and readme
- added output from delete operation
- Added some function descriptions & general improvements
  • Loading branch information
samapriya committed Sep 27, 2023
1 parent 7a71b69 commit 84a7bb2
Show file tree
Hide file tree
Showing 19 changed files with 325 additions and 136 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

**geeadd** (which stands for **Google Earth Engine** batch asset manager with **addons** ) provides a command-line tool for managing Google Earth Engine assets in batch. It allows you to perform a variety of operations on your assets that add to the existing earthengine command line tool. This includes tools that allows a user to estimate their quota usage, asset sizes, set permissions on assets, move and copy and many others. There are also some functions that are build to provided extended functionality like convert any earth engine app script to the source code and search tool which allows you to search both the GEE official data catalog and the community catalog.

This project has been a labor of love starting a few years ago and current improvements and updates can be seen reflected in v1.0.0 and higher up.
This project has been a lot of work starting a few years ago and current improvements and updates can be seen reflected in v1.0.0 and higher up.

![geeadd_main](https://github.com/samapriya/gee_asset_manager_addon/assets/6677629/727263db-03a9-4224-9bbb-82dffe3dccff)

Expand All @@ -28,4 +28,4 @@ Zenodo. https://doi.org/10.5281/zenodo.8287354
```
</b>

Find the [readme docs here](https://samapriya.github.io/gee_asset_manager_addon/)
Find the [readme docs here](https://geeadd.geetools.xyz)
9 changes: 9 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Changelog

### v1.0.0
- Gives better parsing of tasking info
- Search by task ID now
- Now outputs EECU and path for tasks that have those fields
- Fixed size estimation for image vs image collection
- Enhanced user docs and readme
- added output from delete operation
- Added some function descriptions & general improvements

### v0.6.0
- Updated to use API v1 with some updates to avoid breaking changes
- Updated some core tools like size update and quota updation
Expand Down
6 changes: 4 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,17 @@

**geeadd** (which stands for **Google Earth Engine** batch asset manager with **addons** ) provides a command-line tool for managing Google Earth Engine assets in batch. It allows you to perform a variety of operations on your assets that add to the existing earthengine command line tool. This includes tools that allows a user to estimate their quota usage, asset sizes, set permissions on assets, move and copy and many others. There are also some functions that are build to provided extended functionality like convert any earth engine app script to the source code and search tool which allows you to search both the GEE official data catalog and the community catalog.

This project has been a labor of love starting a few years ago and current improvements and updates can be seen reflected in v1.0.0 and higher up.
This project has been a lot of work starting a few years ago and current improvements and updates can be seen reflected in v1.0.0 and higher up.

![geeadd_main](https://github.com/samapriya/gee_asset_manager_addon/assets/6677629/727263db-03a9-4224-9bbb-82dffe3dccff)
![geeadd_main](https://github.com/samapriya/gee_asset_manager_addon/assets/6677629/bf795033-5d5f-40fb-b767-ca87379b9cce)

Like, share and support the Github project. And you can now cite it too

<b>

```
Samapriya Roy. (2023). samapriya/gee_asset_manager_addon: GEE Asset Manager with Addons (0.6.0).
Zenodo. https://doi.org/10.5281/zenodo.8287354
```

</b>
25 changes: 13 additions & 12 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,30 @@
We assume Earth Engine Python API is installed and EE authorised as desribed [here](https://developers.google.com/earth-engine/python_install). From v0.3.4 onwards geeadd will only run on Python 3. Also with the new changes to the Earth Engine API library, the tool was completely modified to work with earthengine-api v0.1.127 and higher. Authenticate your earth engine client by using the following in your command line or terminal setup.

<b>

```
earthengine authenticate
```

</b>
Quick installation **```pip install geeadd```** or **```pip install geeadd --user```**

To install using github:
Quick installation

<b>
```
git clone https://github.com/samapriya/gee_asset_manager_addon
cd gee_asset_manager_addon && pip install -r requirements.txt
python setup.py install
pip install geeadd
pip install geeadd --user
```
</b>

The advantage of having it installed is being able to execute geeadd as any command line tool. I recommend installation within virtual environment. To install run

To get always fresh install using GitHub (**This could be a staging version and will include a pop up on top to remind you of that**)

<b>
```
python setup.py develop or python setup.py install

In a linux distribution
sudo python setup.py develop or sudo python setup.py install
```
pip install git+https://github.com/samapriya/gee_asset_manager_addon.git
```

</b>

The advantage of having it installed is being able to execute geeadd as any command line tool. I recommend installation within virtual environment.
13 changes: 11 additions & 2 deletions docs/projects/access.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
# Change asset permissions

This tool allows you to set asset access for either folder , collection or image recursively meaning you can add collection access properties for multiple assets at the same time. Updates to v1.0.0 and later does not require user email parsing so you can pass it an individual user email, a google group or a google service account and it should work without your needing to specify type.
The Access tool in geeadd empowers users to efficiently manage asset access permissions within Google Earth Engine (GEE). This tool streamlines the process of setting access properties for folders, collections, or images, and it does so recursively. This means you can apply access configurations to multiple assets simultaneously, saving you valuable time and effort.

![geeadd_access](https://user-images.githubusercontent.com/6677629/80338721-0c53a100-882b-11ea-9475-e210ea701433.gif)
#### Key Features

- **Recursive Access Configuration**: The Access tool enables you to apply access permissions recursively. This functionality allows you to set access properties for multiple assets at once, ensuring efficient and consistent management of your Earth Engine assets.

- **Simplified User Identification**: Starting from version 1.0.0 and onwards, the Access tool eliminates the need for manual email parsing. You can now directly provide an individual user email, a Google group, or a Google service account without specifying the type. This enhancement streamlines the process and makes it more user-friendly.

#### Usage
Using the Access tool is straightforward. Simply call the function and provide the necessary arguments.

![geeadd_access](https://github.com/samapriya/gee_asset_manager_addon/assets/6677629/54954596-4583-4b56-a9ac-54b33fef8631)

```
> geeadd access -h
Expand Down
40 changes: 32 additions & 8 deletions docs/projects/app2script.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,41 @@
# GEE app source
# GEE App to Script tool

This tool writes out or prints the underlying earthengine code for any public earthengine app. The tool has an option to export the code into a javascript file that you can then paste into Google Earth Engine code editor.
The App to Script tool in geeadd offers a convenient way to extract the underlying Earth Engine code from any public Earth Engine application. This tool provides two options: it can either print out the code or export it into a JavaScript file for easy integration into the Google Earth Engine code editor.

![geeadd_app2script](https://user-images.githubusercontent.com/6677629/80331908-59c61300-8817-11ea-8075-fb0095f91dab.gif)
#### Key Features

Simple setup can be
- **Code Extraction**: The tool seamlessly retrieves the Earth Engine code that powers any public Earth Engine application, providing users with a clear understanding of the underlying code structure.

```
geeadd app2script --url "https://gena.users.earthengine.app/view/urban-lights"
```
- **JavaScript Export**: Users have the option to export the code directly into a JavaScript file. This file can be opened in any text editor, allowing for easy modification and integration into the Google Earth Engine code editor.

or write to a javascript file which you can then open with any text editor and paste in earthengine code editor
#### Usage

Using the App to Script tool is straightforward. Simply provide the URL of the public Earth Engine application you want to extract the code from.

```bash
# Print out the Earth Engine code
geeadd app2script --url "https://gena.users.earthengine.app/view/urban-lights"
```

```bash
# Export the code to a JavaScript file
geeadd app2script --url "https://gena.users.earthengine.app/view/urban-lights" --outfile "Full path to javascript.js"
```

- `--url`: The URL of the public Earth Engine application.

- `--outfile`: (Optional) The full path to the JavaScript file where you want to export the code.

![GEE App to Script](https://user-images.githubusercontent.com/6677629/80331908-59c61300-8817-11ea-8075-fb0095f91dab.gif)

#### Example

The following example demonstrates how to use the App to Script tool to extract and export Earth Engine code:

```bash
# Print out the Earth Engine code
geeadd app2script --url "https://gena.users.earthengine.app/view/urban-lights"
```



34 changes: 29 additions & 5 deletions docs/projects/cancel_tasks.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,25 @@
# Cancel tasks
# Cancel Tasks tool

This is a simple tool to cancel tasks with specific controls. This allows you to cancel all tasks, all running tasks, all ready tasks or just a single task with a task id.
The Cancel Tasks tool in geeadd is a powerful utility designed to streamline task management within Google Earth Engine (GEE). This tool provides users with precise control over task cancellation, allowing for the cancellation of various task types based on specific criteria. Whether you need to cancel all tasks, terminate running tasks, clear pending tasks, or selectively cancel a single task using its unique task ID, the Cancel Tasks tool has you covered.

![geeadd_tasks_cancel](https://user-images.githubusercontent.com/6677629/80340449-a6691880-882e-11ea-8683-aad1fc2e81c0.gif)
#### Key Features

```
> geeadd cancel -h
- **Efficient Task Cancellation**: The Cancel Tasks tool simplifies the process of terminating GEE tasks, ensuring efficient resource management.

- **Cancel All Tasks**: Users can opt to cancel all active tasks, providing a quick and comprehensive way to clear ongoing processes.

- **Terminate Running Tasks**: For situations requiring the immediate cessation of running tasks, this tool enables the cancellation of running tasks specifically.

- **Clear Pending Tasks**: Pending tasks can be removed in one fell swoop, ensuring that resources are not tied up unnecessarily.

- **Selective Task Cancellation**: Users have the flexibility to target a single task for cancellation by providing its unique task ID.

#### Usage

Utilizing the Cancel Tasks tool is straightforward, and it provides fine-grained control over task cancellation.

```bash
geeadd cancel -h
usage: geeadd cancel [-h] --tasks TASKS

optional arguments:
Expand All @@ -15,3 +29,13 @@ Required named arguments.:
--tasks TASKS You can provide tasks as running or pending or all or even a
single task id
```

- `state`: (Optional) Specifies the state of tasks to cancel (e.g., 'RUNNING' for running tasks, 'READY' for pending tasks).

- `task_id`: (Optional) The unique identifier of the task you want to cancel.

#### Example

Here's an example demonstrating how to use the Cancel Tasks tool to efficiently manage Earth Engine tasks:

![Cancel GEE Tasks](https://user-images.githubusercontent.com/6677629/80340449-a6691880-882e-11ea-8683-aad1fc2e81c0.gif)
37 changes: 27 additions & 10 deletions docs/projects/copy.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,35 @@
# Copy assets
# Copy Assets tool

This script allows us to recursively copy entire folders, collections, images or tables. If you have read acess to assets from another user this will also allow you to copy assets from their assets.
The Copy Assets tool in geeadd offers a versatile solution for copying Earth Engine assets efficiently. With its recursive capabilities, this tool empowers users to duplicate entire folders, collections, images, or tables seamlessly. Additionally, it allows users with read access to assets from other users to copy assets from their repositories, enhancing collaboration and asset management.

![geeadd_copy](https://user-images.githubusercontent.com/6677629/80337918-183e6380-8829-11ea-8482-7359e88fdd75.gif)
#### Key Features

- **Comprehensive Asset Duplication**: The Copy Assets tool enables users to perform recursive copying of Earth Engine assets, ensuring that entire hierarchies of assets can be effortlessly duplicated.

- **User-Friendly Interface**: The tool offers a straightforward command-line interface, making it accessible to users of all experience levels.

- **Copy Assets from Other Users**: Users with read access to assets from other users can utilize this tool to copy assets from their repositories, facilitating collaborative projects and data sharing.

#### Usage

Using the Copy Assets tool is simple and intuitive, allowing users to specify the source and destination paths for asset duplication.

```bash
geeadd copy --initial "existing_asset_path" --final "new_asset_path"
```
geeadd copy -h
usage: geeadd copy [-h] [--initial INITIAL] [--final FINAL]

optional arguments:
-h, --help show this help message and exit
- `--initial`: The existing path of the assets you want to copy.

Required named arguments.:
--initial INITIAL Existing path of assets
--final FINAL New path for assets
- `--final`: The new path where you want to duplicate the assets.

#### Example

Here's an example demonstrating how to use the Copy Assets tool to duplicate Earth Engine assets:

```bash
geeadd copy --initial "users/your_username/your_collection" --final "users/your_username/copied_collection"
```

![Copy GEE Assets](https://user-images.githubusercontent.com/6677629/80337918-183e6380-8829-11ea-8482-7359e88fdd75.gif)

The Copy Assets tool in geeadd simplifies asset duplication and promotes efficient asset management within Google Earth Engine.
32 changes: 22 additions & 10 deletions docs/projects/delete.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,29 @@
# Delete Assets
# Delete Assets tool

The delete is recursive, meaning it will delete also all children assets: images, collections and folders. Use with caution!
The Delete Assets tool in GEEadd is a powerful utility designed to facilitate asset management within Google Earth Engine (GEE). This tool empowers users to perform recursive deletions of Earth Engine assets, including folders, collections, images, and their child assets. However, it is important to exercise caution while using this tool, as it permanently removes assets and their associated data.

![geeadd_delete](https://user-images.githubusercontent.com/6677629/80338936-9d2a7c80-882b-11ea-948e-20baf061a2f2.gif)
#### Key Features

- **Comprehensive Asset Deletion**: The Delete Assets tool allows users to perform recursive deletions of assets, ensuring that entire hierarchies of assets can be removed with a single command.

- **Use with Caution**: Due to the recursive nature of this tool, it will delete not only the specified asset but also all its child assets, including images, collections, and folders. Therefore, it is essential to use this tool with caution to avoid unintentional data loss.

#### Usage

Using the Delete Assets tool is straightforward, requiring only the specification of the target Earth Engine asset for deletion.

```bash
geeadd delete --id "asset_path_to_delete"
```
> geeadd delete -h
usage: geeadd delete [-h] --id ID

optional arguments:
-h, --help show this help message and exit
- `--id`: The full path to the asset you want to delete. This tool will recursively remove all child assets, including images, collections, and folders associated with the specified asset.

Required named arguments.:
--id ID Full path to asset for deletion. Recursively removes all
folders, collections and images.
#### Example

Here's an example demonstrating how to use the Delete Assets tool to remove an Earth Engine asset and all its child assets:

```bash
geeadd delete --id "users/your_username/your_collection"
```

![Delete GEE Assets](https://user-images.githubusercontent.com/6677629/80338936-9d2a7c80-882b-11ea-948e-20baf061a2f2.gif)
33 changes: 22 additions & 11 deletions docs/projects/delete_metadata.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,29 @@
# Delete asset metadata
# Delete metadata tool

This tool allows you to delete a specific property across a metadata. This is useful to reset any property for an ingested collection.
The Asset Metadata Delete tool in geeadd provides users with a valuable capability to delete specific properties from metadata associated with Earth Engine assets. This tool is particularly useful when you need to reset or remove a property value from the metadata of an ingested collection, image, or table.

![geeadd_delete_metadata](https://user-images.githubusercontent.com/6677629/80341015-a9b0d400-882f-11ea-84ad-d7ac46798cc7.gif)
#### Key Features

- **Selective Metadata Property Deletion**: The Asset Metadata Delete tool allows users to selectively delete a specific property from the metadata associated with an Earth Engine asset.

#### Usage

Using the Asset Metadata Delete tool is straightforward, requiring the specification of the target Earth Engine asset and the property to be deleted from its metadata.

```bash
geeadd delete_metadata --asset "asset_path_here" --property "metadata_property_to_delete"
```
> geeadd delete_metadata -h
usage: geeadd delete_metadata [-h] --asset ASSET --property PROPERTY

optional arguments:
-h, --help show this help message and exit
- `--asset`: The path to the Earth Engine asset from which you want to remove a specific metadata property.

- `--property`: The name of the metadata property that you want to delete.

Required named arguments.:
--asset ASSET This is the path to the earth engine asset whose
permission you are changing collection/image
--property PROPERTY Metadata name that you want to delete
#### Example

Here's an example illustrating how to use the Asset Metadata Delete tool to remove a specific property from the metadata of an Earth Engine asset:

```bash
geeadd delete_metadata --asset "users/your_username/your_collection" --property "description"
```

![Delete Metadata GEE Asset](https://user-images.githubusercontent.com/6677629/80341015-a9b0d400-882f-11ea-84ad-d7ac46798cc7.gif)
Loading

0 comments on commit 84a7bb2

Please sign in to comment.