Skip to content

Commit

Permalink
docs: reflect new configuration setup
Browse files Browse the repository at this point in the history
Signed-off-by: Panos Vagenas <[email protected]>
  • Loading branch information
vagenas committed Jul 27, 2023
1 parent e616cd8 commit 0773596
Show file tree
Hide file tree
Showing 9 changed files with 94 additions and 101 deletions.
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
*Interact with the Deep Search platform for new knowledge explorations and discoveries*


The Deep Search Toolkit is a Python SDK allowing a user to interact with the Deep Search platform. The toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.
The Deep Search Toolkit is a Python SDK and CLI allowing users to interact with the Deep Search platform.
The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.


[Learn about IBM Deep Search](https://ds4sd.github.io/)
Expand Down Expand Up @@ -53,14 +54,14 @@ Python 3.8+

## Start using the toolkit

### Login to Deep Search
For details, check the [authentication docs](https://ds4sd.github.io/deepsearch-toolkit/getting_started/#authentication).
### Set up a profile
For details, check [Profiles](https://ds4sd.github.io/deepsearch-toolkit/guide/configuration#profiles).
```console
deepsearch login
deepsearch profile config
```

### Convert a document
For details, check the [document conversion docs](https://ds4sd.github.io/deepsearch-toolkit/guide/convert_doc).
For details, check [Document conversion](https://ds4sd.github.io/deepsearch-toolkit/guide/convert_doc).
```console
deepsearch documents convert -p 1234567890abcdefghijklmnopqrstvwyz123456 -u https://arxiv.org/pdf/2206.00785.pdf
```
Expand Down
8 changes: 3 additions & 5 deletions deepsearch/model/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,12 @@ app.run(host="127.0.0.1", port=8000)

### Settings
App configuration is done in [`Settings`](server/config.py) based on
[Pydantic settings management](https://docs.pydantic.dev/latest/usage/settings/).
[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/).

E.g. the required API key can be set via env var `DS_MODEL_API_KEY` (for precedence rules,
check the
[Pydantic docs](https://docs.pydantic.dev/latest/usage/settings/#field-value-priority)).
E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`.

### OpenAPI
The OpenAPI UI is served under `/docs`, e.g. http://127.0.0.1:8000/docs.
The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs.

## Developing a new model
To develop a new model class for an existing [kind](kinds/), inherit from the base model
Expand Down
20 changes: 9 additions & 11 deletions docs/getting_started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,26 @@ After registering with [Deep Search](https://ds4sd.github.io/), you can obtain y


---
### Configure a profile
### Set up a profile

=== "CLI"
Configure a profile using [`deepsearch profile config`](../cli-reference.md#profile).
Set up a [profile][profiles] using `deepsearch profile config`.
In its basic form it looks like this:
<div class="termy">

```console
$ deepsearch profile config

Host: # (1)
Username: # (2)
Api key: # (3)
Host: https://ds.example.com # the Deep Search instance you are using
Username: [email protected] # your username
Api key: (hidden) # your API key
```

</div>

1. Input the Deep Search deployment host to use.
2. Enter your `User name`.
3. Enter your `API key`.

By providing a profile name (via option `--profile-name`) you can configure multiple
different profiles, which you can then easily switch between and manage.

For details, check [Profiles](../guide/configuration.md#profiles).
For details, check [Profiles][profiles].

### Validate the setup

Expand Down Expand Up @@ -89,3 +84,6 @@ Here, we show a simple way to convert documents using [Deep Search](https://ds4s
```

---


[profiles]: ../guide/configuration.md#profiles
7 changes: 2 additions & 5 deletions docs/guide/cli_plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,9 @@ Options:
--help Show this message and exit.
Commands:
config Manage CLI config files
cps Interact with DeepSearch CPS component
[ ... ] [ ... ]
example
login Login to DeepSearch platform
query Interact with DeepSearch Query component
version Print the client and server version information
[ ... ] [ ... ]
```

And running our newly added command gives us...
Expand Down
124 changes: 61 additions & 63 deletions docs/guide/configuration.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
# Configuration

The Toolkit can be configured both via the CLI and via
environment variables.
The Toolkit can be configured via the CLI and via environment variables.

Besides *global* settings, the Toolkit also allows the configuration of multiple
*profiles* for enabling users to easily work with different Deep Search deployments.

## Profiles

The Toolkit provides the capability of easily interacting with different Deep Search
instances through the use of *profiles*. A user may define multiple profiles, identified
by profile name, and can easily switch between them.

### Overview

For an overview of the profile management commands check `deepsearch profile --help`:
Expand All @@ -32,8 +35,8 @@ $ deepsearch profile --help

### Profile setup

To setup a profile use [`deepsearch profile config`](../cli-reference.md#profile),
providing options as needed (for a full reference run with `--help`).
To set up a profile use `deepsearch profile config`, providing options as needed (for a
full reference check `deepsearch profile config --help`).

Here is a basic invocation example:

Expand All @@ -44,76 +47,71 @@ Username: [email protected]
Api key:
```

> NOTE: If you had used the meanwhile deprecated `deepsearch login` command to set up a
config file in the default location, that will automatically be migrated to a profile.
!!! note
If you had used the meanwhile deprecated `deepsearch login` command to set up a
config file in the default location, that will automatically be migrated to a profile.

### Usage

Here some usage examples:

=== "CLI"

```console
$ # configure a profile
$ deepsearch profile config --profile-name foo --host ds-1.example.com ...
$ # -> "foo" now available and selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
$
$ # configure another profile
$ deepsearch profile config --profile-name bar --host ds-2.example.com ...
$ # -> "bar" now available and selected as the active profile
$
$ deepsearch profile list
$ # -> displays all profiles, with "bar" marked as the active one
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "bar"
$
$ # switch to previous profile
$ deepsearch profile use foo
$ # -> "foo" now selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
```

=== "Python"

```python
from deepsearch.cps.client.api import CpsApi

# using the active profile; let's assume that is "foo"
api = CpsApi.from_env()
print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "foo"

# using a profile by name
api = CpsApi.from_env(profile_name="bar")
print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "bar"
```

---
#### Usage in CLI

```console
$ # configure a profile
$ deepsearch profile config --profile-name foo --host ds-1.example.com ...
$ # -> "foo" now available and selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
$
$ # configure another profile
$ deepsearch profile config --profile-name bar --host ds-2.example.com ...
$ # -> "bar" now available and selected as the active profile
$
$ deepsearch profile list
$ # -> displays all profiles, with "bar" marked as the active one
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "bar"
$
$ # switch to previous profile
$ deepsearch profile use foo
$ # -> "foo" now selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
```

#### Usage in Python

To use the active profile (recommended usage pattern):
```python
from deepsearch.cps.client.api import CpsApi

api = CpsApi.from_env()

print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to active profile
```

To use a specific profile:
```python
from deepsearch.cps.client.api import CpsApi

api = CpsApi.from_env(profile_name="foo")

print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "foo"
```

## Environment variables

Under the hood, configuration management in the Toolkit is implemented based on [Pydantic
Settings with dotenv support](https://docs.pydantic.dev/1.10/usage/settings).

Configuration is persisted in Toolkit-managed dotenv files, but can be overriden via
Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings).
Besides allowing the Toolkit to easily persist and manage config options (e.g. as seen
above for profiles), this also means that configuration can be easily overriden via
environment variables.

To see which environment variables are supported, check the relevant [Pydantic Settings
classes][settings_file], also taking into account any defined prefixes.

For example you can set the profile for the scope of a single operation (e.g. a project
listing) as follows:

```console
DEEPSEARCH_PROFILE=bar deepsearch cps projects list
```

[settings_file]: https://github.com/DS4SD/deepsearch-toolkit/blob/main/deepsearch/core/client/settings.py
8 changes: 4 additions & 4 deletions docs/guide/data_indices.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Suppose you want to create an index called `NAME`. Optionally, a description,`DE
</div>
=== "Python"

After you have generated the api object (from [login configuration](../installation/login.md)),
After you have generated the api object (from a [profile](../configuration#usage-in-python)):

```python
api.data_indices.create(proj_key=PROJ_KEY, name=NAME, desc=DESC)
Expand Down Expand Up @@ -159,9 +159,9 @@ Attachments can be added to an index item in a project. Briefly, attachments hav
attachment_key = "usr_my_attachment" # optional. if set need start with 'usr_' and be snake_case

index.add_item_attachment(
api=api,
index_item_id=index_item_id,
attachment_path=attachment_path,
api=api,
index_item_id=index_item_id,
attachment_path=attachment_path,
attachment_key=attachment_key, # optional
)
```
6 changes: 3 additions & 3 deletions docs/guide/kgs.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,20 +64,20 @@ More example queries are available in the [examples gallery](../gallery/index.md
$ deepsearch cps kgs download -p 'abc123' -k 'zxc987'

url
--------------------------------
--------------------------------
https://host.domain/download-url
```

</div>

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication))
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)):

```python
proj_key = "abc123" # get your proj_key from the UI details
kg_key = "xyz987" # get kg_key (also called bag_key) from the UI details
kg = api.knowledge_graphs.get(proj_key, kg_key)

download_url = kg.download()
```
```
7 changes: 3 additions & 4 deletions docs/guide/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo
## Project management

!!! info
Make sure to [configure your login](../getting_started/index.md#authentication) before
using the project management features listed below.
Make sure to [complete your profile setup](../getting_started/index.md#set-up-a-profile) before using the project management features listed below.

### Creating a project

Expand All @@ -39,7 +38,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication)), creating a project is very easy.
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)), creating a project is very easy.

```python
proj = api.projects.create(name="my-project")
Expand Down Expand Up @@ -70,7 +69,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication)), listing projects is very easy.
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)), listing projects is very easy.

```python
projects = api.projects.list() # returns list of projects
Expand Down
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@

*Interact with the Deep Search platform for new knowledge explorations and discoveries*

The Deep Search Toolkit provides a Python SDK and a CLI allowing a user to interact with the Deep Search platform. The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.
The Deep Search Toolkit is a Python SDK and CLI allowing users to interact with the Deep Search platform.
The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.


[Learn about IBM Deep Search :octicons-link-external-16:](https://ds4sd.github.io/){ .md-button }

Expand Down

0 comments on commit 0773596

Please sign in to comment.