BioDex Docs

About

BioDex is a species classification tool for Taxonomists & Collection workers developed by ETH Library Lab.

The project consists of three parts/repos

1. mobile app for photographing specimens and viewing results
2. prediction API for classifying images
3. prediction model for training new models and data processing workflow

See more general info at biodex.ethz.ch/about/

Contents

1. Project Overview
2. Prediction API & Website
3. Mobile App & API
4. Image Repo (Docker Hub)
5. Project Maintenance & Contribution
6. Additional Info
   • Taxonomy
   • Podman
   • Copyright
   • Django Basic Queries

Project Overview

Responsibility Matrix & Contact guidelines

This project is kindly hosted by ETH Library. Training new prediction models, improvements to the app or general any development work relies on contributions from contributors.

For general enquiries

[email protected]  

questions about Entomology, use in collections or discussing research colloborations

[email protected]

technical enquiries, prediction models, classification techniques etc.

[email protected]

if there is website or api services not available

[email protected]  

For general bugs or requesting new features, please submit an issue to the relevant github repo, or better yet, submit a pull request.

Responsibilites

Task	ETH Library	ETH EC	Community
Hosting	✓
Deployment	✓
Development			✓
Approving Pull Requests			✓
Prediction Model Training			✓
User Account Administration		✓
Replying to Contact Emails		✓

<<<<<<< Updated upstream

About

BioDex is a species classification tool for Taxonomists & Collection workers developed by ETH Library Lab.

See more at biodex.ethz.ch/about/

Contents

1. Project Overview
2. Prediction API & Website
3. Mobile App & API
4. Image Repo (Docker Hub)
5. Project Maintenance & Contribution
6. Additional Info
   • Taxonomy
   • Podman
   • Copyright
   • Django Basic Queries

Project Overview

=======

Stashed changes

Prediction API & Website

The Prediction API is built with Django REST Framework. It is used to:

• store image and taxonomic data for model training
• act as a gateway to the prediction model (by preprocessing posted images and postprocessing model response results)
• gather response metadata (e.g. example images)
• store data from prediction requests to be used for future analysis

Detailed README

Mobile App & API

The mobile API is not in this codebase, for a detailed readme see that repo.

Data

data processing is performed in a seperate repo

Prediction Model

model training is performed in a seperate repo

Maintenance and Contribution

This Project is kindly being hosted ETH Library. Developing new features and bug fixing is open to the community. We use Github Issues to track requests and bugs. If you would like to contribute significant changes please get in contact.

Additional Info

This section includes some background information and useful commands for some of the frameworks used on this project.

Taxonomy

binomial naming

species are named using the binomial format where the combination of the Genus and specific epithet define a species, e.g. Homo sapiens.

sp.

sp. is used when the genus can be identified but the exact species cannot or does not need to be determined. e.g. Homo sp. refers to some unidentified species of the genus Homo.

Copyright

When using images from other people/collections/entities it is very important to ensure that we have permission to distribute the images.

The BioDex project is distributed under a creative commons licence; Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) http://creativecommons.org/licenses/by-nc-sa/3.0/

Podman

open an interactive shell on a running container

podman exec -it <container-number> /bin/bash

save local docker image and transfer to server

save docker image as a tar file

docker save -o <path to generated file.tar> <image name>

Then copy your image to the host server with regular file transfer tools such as cp, scp.
Then load the image into Docker:

docker load -i <path to image tar file>

using docker hub

first login to dockerhub biodex in the cmd line

docker login

images can then be pushed or pulled from the docker hub repo for example;

docker push biodex/main:prediction_model__201911171137

note that there is only one private repo in the free tier so version tagging is added after double underscore instead of colon

Rebuild and Run for Production

generate new secret keys

from django.core.management.utils import get_random_secret_key
print(get_random_secret_key())

create an admin/superuser in the django app

podman exec -it WEB-CTR-NAME python manage.py createsuperuser

load fixture files using the convenience bash script

podman exec web sh load_fixtures.sh

connect to an interactive shell for the postgres database

podman exec DB-CTR-NAME psql --username=admin --dbname=biodex_dev

Authentication

Uses Djoser

list of all available endpoints https://djoser.readthedocs.io/en/latest/getting_started.html#available-endpoints

token endpoints https://djoser.readthedocs.io/en/latest/token_endpoints.html

create user

http://127.0.0.1:8000/api/auth/users/

request token

http://127.0.0.1:8000/api/auth/token/login/

Example Curl Requests

Request Token

curl -X POST -d '{"username": "admin","password": "1234"}' -H 'Content-Type: application/json' http://127.0.0.1:8000/api/auth/token/login/

Using Token

curl -X POST http://127.0.0.1:8000/api/predict/ -H 'Authorization: Token a21e26bd12a16542f940d641e840e32ad16a26d0' [{"id":1,"name":"admin"]

django basic queries

get the first 5 records from a model

qryset = Image.objects.all()[:5]

display the field names and values for those records

qryset.values()

look up fields in a related model/table (i.e. join) connect the field in the current model to the desired field in the other model

cls.values('species_key','image_key__image')